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Preface 


In  response  to  concerns  about  the  elevated  rate  of  suicide  among  U.S.  service  members,  the 
congressionally  mandated  Task  Force  on  the  Prevention  of  Suicide  by  Members  of  the  Armed 
Forces  offered  a  series  of  recommendations  to  help  strengthen  the  U.S.  Department  of  Defense 
(DoD)  suicide  prevention  programs.  The  task  force’s  final  recommendation  was  for  DoD  to 
“create  a  unified,  strategic,  and  comprehensive  DoD  plan  for  research  in  military  suicide  pre¬ 
vention  ensuring  that  the  DoD’s  military  suicide  prevention  research  portfolio  is  thoughtfully 
planned  to  cover  topics  in  prevention,  intervention,  and  postvention”  (U.S.  Department  of 
Defense  Task  Force  on  the  Prevention  of  Suicide  by  Members  of  the  Armed  Forces,  2010, 
p.  ES-18). 

The  RAND  Corporation  was  asked  to  provide  guidance  that  DoD  can  use  to  develop 
this  recommended  unified,  strategic,  and  comprehensive  plan.  The  study  was  organized  around 
three  overarching  research  aims:  (1)  catalog  research  being  conducted  on  suicide  prevention 
that  is  directly  relevant  to  military  personnel,  (2)  examine  whether  current  research  maps 
onto  DoD’s  strategic  research  needs  related  to  suicide  prevention,  and  (3)  ensure  that  any  pro¬ 
posed  DoD  research  strategy  aligns  with  the  national  research  strategy  and  is  integrated  with 
DoD’s  data,  surveillance,  and  program  evaluation  strategies.  RAND  took  a  multidisciplinary 
approach  to  meeting  these  three  aims,  drawing  from  the  disciplines  of  psychology,  epidemiol¬ 
ogy,  statistics,  and  economics.  This  report  presents  the  results  of  the  study.  It  should  be  of  inter¬ 
est  to  policy  officials  charged  with  implementing  suicide  prevention  programs,  analysts  who 
compile  suicide  prevention  research  portfolios  for  program  evaluation  or  other  purposes,  and 
others  who  are  engaged  in  ensuring  a  comprehensive  response  to  suicide  among  service  mem¬ 
bers,  including  members  of  Congress  and  military  and  veteran  service  organizations. 

This  research  was  sponsored  by  the  Office  of  the  Under  Secretary  of  Defense  for  Per¬ 
sonnel  and  Readiness  and  conducted  within  the  Forces  and  Resources  Policy  Center  of  the 
RAND  National  Defense  Research  Institute,  a  federally  funded  research  and  development 
center  sponsored  by  the  Office  of  the  Secretary  of  Defense,  the  Joint  Staff,  the  Unified  Com¬ 
batant  Commands,  the  Navy,  the  Marine  Corps,  the  defense  agencies,  and  the  defense  Intel¬ 
ligence  Community. 

For  more  information  on  the  RAND  Forces  and  Resources  Center,  see  http://www.rand. 
org/nsrd/ndri/centers/frp.html  or  contact  the  director  (contact  information  is  provided  on  the 
web  page). 
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Summary 


The  U.S.  Department  of  Defense  (DoD)  has  been  increasingly  concerned  about  the  elevated 
rate  of  suicide  among  U.S.  service  members.  At  the  end  of  the  last  decade,  a  report  by  the 
RAND  Corporation  (Ramchand  et  ah,  2011)  and  another  by  the  congressionally  mandated 
Task  Force  on  the  Prevention  of  Suicide  by  Members  of  the  Armed  Forces  offered  a  series  of 
recommendations  to  help  strengthen  DoD’s  suicide  prevention  programs.  The  task  force’s  final 
recommendation  was  for  DoD  to  “create  a  unified,  strategic,  and  comprehensive  DoD  plan 
for  research  in  military  suicide  prevention  ensuring  that  the  DoD’s  military  suicide  prevention 
research  portfolio  is  thoughtfully  planned  to  cover  topics  in  prevention,  intervention,  and  post- 
vention”  (U.S.  Department  of  Defense  Task  Force  on  the  Prevention  of  Suicide  by  Members  of 
the  Armed  Forces,  2010,  p.  ES-18). 

Shortly  after  the  task  force  report’s  release,  the  National  Action  Alliance  for  Suicide  Pre¬ 
vention  (NAASP),  a  public-private  partnership  charged  with  advancing  the  National  Strat¬ 
egy  for  Suicide  Prevention,  established  the  Research  Prioritization  Task  Force  to  develop  “an 
agenda  for  research  with  the  stated  goal  to  reduce  morbidity  (attempts)  and  mortality  (deaths), 
each  by  at  least  20%  in  five  years  and  40%  or  greater  in  10  years,  if  implemented  fully  and 
successfully’  (NAASP,  2014,  p.  7;  emphasis  in  original).  This  report,  produced  by  the  RAND 
National  Defense  Research  Institute,  represents  an  effort  to  assist  DoD  in  creating  a  strategic 
research  plan  that  aligns  with  the  national  research  agenda. 

This  study’s  objective  was  to  provide  guidance  that  DoD  could  use  to  develop  the  rec¬ 
ommended  unified,  strategic,  and  comprehensive  plan.  The  study  was  organized  around 
three  overarching  research  aims:  (1)  catalog  research  being  conducted  on  suicide  prevention 
that  is  directly  relevant  to  military  personnel,  (2)  examine  whether  current  research  maps 
onto  DoD’s  strategic  research  needs  related  to  suicide  prevention,  and  (3)  ensure  that  any 
proposed  DoD  research  strategy  aligns  with  the  national  research  strategy  and  is  integrated 
with  DoD’s  data,  surveillance,  and  program  evaluation  strategies.  We  took  a  multidisciplinary 
approach  to  achieving  these  three  aims,  drawing  from  the  disciplines  of  psychology,  epidemiol¬ 
ogy,  statistics,  and  economics. 


What  Suicide  Prevention  Research  Being  Done  in  the  United  States  Is 
Directly  Relevant  to  Military  Personnel? 

We  conducted  a  comprehensive  scan  to  identify  who  in  the  United  States  is  funding  research 
on  suicide  prevention  that  is  relevant  to  military  personnel  and  to  catalog  exactly  what  they 
were  funding.  We  found  that  DoD  is  the  largest  single  funder  of  suicide  prevention  research. 
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having  recently  funded  61  studies  at  a  cost  of  more  than  $100  million.  This  figure  does  not 
include  the  Army  Study  to  Assess  Risk  and  Resilience  in  Servicememhers  (known  as  Army 
STARRS),  a  $65  million  study  funded  jointly  hy  the  U.S.  Army  and  the  National  Institute  of 
Mental  Health.  While  DoD  is  the  largest  funder  of  suicide  prevention  research  with  relevance 
to  military  personnel  in  the  United  States,  this  total  represents  a  little  less  than  one-third  of  all 
relevant  studies,  with  the  U.S.  Department  of  Health  and  Human  Services  (primarily  through 
the  National  Institutes  of  Health),  the  U.S.  Department  of  Veterans  Affairs  (VA),  and  the 
American  Foundation  for  Suicide  Prevention  funding  the  remaining  two-thirds. 

The  results  focused  primarily  on  DoD  and  overall  trends  relative  to  12  NAASP  catego¬ 
ries  representing  “aspirational  goals”  for  suicide  prevention  and  seven  Military  Operational 
Medicine  Research  Program  (MOMRP)  categories  that  make  up  MOMRP’s  continuum  of 
care  framework.  With  respect  to  the  NAASP  categories,  current  suicide  prevention  research 
funded  hy  DoD  and  other  funders  tends  to  focus  on  studies  that  attempt  to  identify  who  dies 
hy  suicide  (i.e.,  risk  and  protective  factor  interactions),  psychotherapeutic  interventions  to  treat 
individuals  at  risk  for  suicide  (i.e.,  psychosocial  interventions),  and  ensuring  that  those  at  risk 
can  access  affordable,  accessible,  and  effective  care.  We  based  this  assessment  on  the  number  of 
studies  funded  and  the  total  amount  of  funding  devoted  to  these  areas  of  research. 

In  contrast,  relatively  few  studies  funded  by  DoD  and  other  entities  focus  on  preventing 
reattempts  among  those  who  have  previously  attempted  suicide  or  the  reduction  in  access  to 
lethal  means.  These  were  also  areas  for  which  there  were  relatively  low  amounts  of  funding. 
Interestingly,  although  prevention  was  an  area  of  low  overall  spending,  DoD’s  highest  level 
of  mean  funding  per  study  was  for  studies  on  the  prevention  of  reattempts.  Studies  on  access 
to  lethal  means  are  particularly  underfunded  across  funders  compared  with  the  rest  of  the 
research  portfolio,  perhaps  because  of  legislation  that  greatly  restricted  this  type  of  research 
or  because  of  the  perceived  lack  of  cultural  acceptability  of  means  reduction  as  an  effective 
component  of  military  suicide  prevention.  It  is  also  notable  that  DoD  funded  fewer  studies  of 
biological  interventions,  compared  with  other  funders,  but  it  still  accounted  for  65  percent 
of  all  funding  in  this  category. 

Accounting  for  studies  according  to  the  MOMRP  categories  shows  that  most,  by  far, 
both  overall  and  by  DoD  specifically,  are  being  conducted  on  who  is  at  risk  of  dying  by  suicide 
(i.e.,  epidemiology  or  basic  science/neurological  mechanisms).  There  are  also  a  large  number  of 
studies  focusing  on  treatment  and  prevention  training/education.  Similarly,  the  most  funding 
by  far  is  being  spent  on  epidemiology/basic  science/neurological  mechanisms  and  treatment, 
with  a  moderate  amount  of  spending  on  prevention  education  and  training  and  on  assessment. 
Among  DoD  studies,  mean  funding  per  study  was  highest  for  studies  of  treatment,  epidemiol¬ 
ogy  and  basic  science,  and — surprisingly — postvention,  despite  the  low  level  of  overall  funding 
going  to  studies  on  that  topic.  Indeed,  relatively  few  studies  are  examining  either  postvention 
or  recovery  and  return  to  duty,  and  there  is  very  little  spending  by  non-DoD  funders  in  these 
areas  as  well,  suggesting  that  if  DoD  does  value  research  in  these  areas,  it  will  likely  have  to 
fund  the  requisite  studies.  Indeed,  recovery  and  return  to  duty  may  be  particularly  central  to 
DoD’s  mission  (despite  DoD’s  relatively  low  funding  levels  for  research  in  this  area),  but  it  may 
not  be  not  viewed  as  a  key  area  of  research  by  other  funders. 

These  results  concerning  research  priorities  should  be  interpreted  with  the  caveat  that 
they  do  not  include  the  major  Army  STARRS  study,  for  which  the  Army  has  allocated 
$50  million  (of  $65  million  in  total  funding  for  the  study)  to  examine  11  of  12  NAASP  cat¬ 
egories  and  all  seven  MOMRP  categories  of  suicide  prevention  research.  When  we  report  that 
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funding  is  low  in  a  given  area,  that  assessment  does  not  take  into  account  Army  STARRS 
research. 


Which  Suicide  Prevention  Research  Needs  Are  DoD  Priorities? 

Having  established  what  research  DoD  is  conducting  on  suicide,  we  conducted  an  assessment 
of  DoD’s  suicide  prevention  research  needs  to  identify  the  department’s  priorities.  We  modi- 
hed  12  aspirational  goals  identified  hy  the  NAASP  Research  Prioritization  Task  Force  so  that 
we  could  isolate  the  input  and  priorities  of  DoD  stakeholders  from  those  of  other,  non-DoD 
stakeholders.  Table  S.l  shows  the  modified  set  of  aspirational  goals  and  the  shorthand  used  in 
the  analysis. 

We  used  the  same  procedures  as  the  NAASP  Research  Prioritization  Task  Force,  includ¬ 
ing  identical  online  elicitation  software  and  processes.  We  asked  DoD  stakeholders  to  con¬ 
sider  the  goals’  merits  on  five  grounds:  (1)  overall  importance,  (2)  effectiveness,  (3)  cultural 
acceptability,  (4)  cost,  and  (5)  learning  potential  (i.e.,  the  amount  that  could  be  learned  by 
pursuing  strategies  tied  to  the  goals).  Participants  ranked  the  goals  relative  to  the  criteria 
using  the  online  RAND  ExpertLens™  elicitation  process.  Because  of  lower-than-anticipated 
participation  among  DoD  stakeholders,  we  sought  to  confirm  responses  in  two  domains — 
effectiveness  and  learning  potential — by  implementing  a  modified  expert  elicitation  among 
seven  RAND  researchers,  all  of  whom  have  conducted  recent  research  on  or  related  to  military 
suicide  prevention. 

According  to  these  criteria,  gatekeeper  training  stands  out.  Although  DoD  experts  did 
not  believe  that  this  goal  was  currently  very  effective,  they  ranked  it  as  important,  culturally 
acceptable,  and  the  least  costly.  The  RAND  panel  ranked  research  relating  to  gatekeeper  train¬ 
ing  as  having  the  second  highest  learning  potential.  Furthermore,  prior  research  suggests  that 
these  types  of  trainings  are  widely  used  in  DoD.  Provider  training  and  strategies  for  enhancing 
continuity  of  care  appear  similarly  promising  in  terms  of  their  importance  and  effectiveness; 
however,  the  RAND  panel  ranked  them  as  likely  to  yield  low  returns  in  future  research,  largely 
because  they  believed  that  there  was  not  much  more  to  learn  about  these  approaches. 


How  Do  DoD's  Prioritized  Needs  Map  Against  the  Ongoing  Suicide 
Prevention  Research? 

We  conducted  a  preliminary  gap  analysis  by  combining  the  data  on  research  being  conducted 
and  the  elicited  DoD  prioritized  needs  to  examine  whether  current  research  on  suicide  preven¬ 
tion  that  is  relevant  to  military  personnel — in  terms  of  both  the  number  of  ongoing  studies 
and  the  amount  of  funds  allocated — aligned  with  those  DoD  needs.  We  examined  “needs” 
across  the  five  domains:  importance,  effectiveness,  cultural  acceptability,  cost,  and  potential 
learning.  The  aim  was  to  help  DoD  begin  thinking  about  a  strategic  research  agenda  for  sui¬ 
cide  prevention  research. 

The  results  suggest  there  is  a  gap  between  priorities  and  research.  Specifically,  current 
studies  and  funding  align  best  with  the  domains  of  effectiveness  and  cost.  In  other  words, 
more  studies  and  more  funding  are  going  toward  suicide  prevention  goals  that  are  already 
ranked  as  most  highly  effective,  as  well  as  those  considered  the  most  costly  to  implement. 


xviii  Developing  a  Research  Strategy  for  Suicide  Prevention  in  the  Department  of  Defense 


Table  S.1 

Modified  DoD  Aspirational  Goals 


NAASP  Aspirational  Goal 

DoD  Aspirational  Goal 

RAND  Shorthand 

Population-based  risk-reduction/ 
resilience-building 

Implement  population-based  programs 
that  reduce  suicide  risk  factors  and  build 
resilience. 

Risk  reduction 

Provider  and  gatekeeper  training 

Ensure  non-health  professionals 
(i.e.,  noncommissioned  officers,  chaplains) 
who  come  in  contact  with  suicidal  individuals 
are  trained  to  identify,  care  for,  and  refer 
persons  at  risk. 

Gatekeeper  training 

Provider  and  gatekeeper  training 

Train  health  care  professionals  to  identify 
those  at  risk  for  suicide  and  to  manage  their 
treatment. 

Provider  training 

Stigma  reduction 

Encourage  service  members  and  their 
families  to  be  knowledgeable  about  and 
proactively  seek  treatment. 

Help-seeking 

Affordable,  accessible,  and  effective 
care 

Deliver  high-quality  treatments  for  mental 
illnesses  (e.g.,  depression,  post-traumatic 
stress  disorder)  that  are  associated  with 
suicide. 

Quality  care 

Population-based  screening 

Conduct  population-based  screening  to 
identify  those  at  risk  for  suicide. 

Screening 

Reduction  in  access  to  lethal  means 

Reduce  service  members'  access  to  the 
means  that  they  might  use  to  take  their 
own  lives. 

Reduced  access 

Psychosocial  interventions  for  those 
at  risk 

Improve  psychosocial  interventions  used  by 
clinicians  (e.g.,  psychiatrists,  psychologists, 
social  workers)  to  identify  and  treat  those  at 
risk  for  suicide. 

Psychosocial  interventions 

Improved  biological  interventions 

Identify  biological  interventions  that 
clinicians  could  use  to  treat  suicidal  behavior. 

Biological  interventions 

Prediction  of  imminent  risk 

Develop  strategies  to  predict  which 
individuals  are  at  imminent  risk  of  suicide. 

Prediction 

Enhanced  continuity  of  care 

Achieve  continuity  of  care  between 
providers,  across  installations,  and  with  the 
civilian  and  VA  systems. 

Continuity  of  care 

Prevention  of  reattempts 

Implement  strategies  to  prevent  suicide 
reattempts. 

Prevent  reattempts 

Risk  and  protective  factor 
interactions 

N/A 

NOTE:  Goals  are  worded  verbatim  from  the  RAND  ExpertLens  elicitation  exercise  conducted  for  this  study. 

The  goal  "risk  and  protective  factor  interactions"  was  excluded  because  we  could  not  equate  it  with  a  specific 
aspirational  goal. 


However,  there  is  no  direct  link  between  what  is  being  funded  and  what  DoD  representatives 
perceive  as  important.  Furthermore,  while  there  are  few  studies  and  little  funding  is  being  allo¬ 
cated  to  goals  ranked  as  least  culturally  acceptable,  there  are  also  few  studies  and  little  funding 
allocated  to  goals  ranked  as  most  culturally  acceptable.  Finally,  there  is  an  inverse  relationship 
between  the  number  of  studies  and  the  amount  of  funding  allocated  to  goals  with  the  most 
learning  potential,  with  more  studies  and  funding  going  toward  goals  ranked  as  having  the 
lowest  learning  potential. 
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We  improved  on  the  preliminary  gap  analysis  by  creating  an  econometric  model  that 
uses  the  input  from  the  needs  assessment  and  incorporates  additional  parameters  to  rank  or 
prioritize  research  in  a  way  that  yields  maximum  impact  in  terms  of  reduced  mortality — a 
beneht-cost  index  that  accounts  for  both  the  explicit  benefit  (i.e.,  suicides  prevented)  and  the 
explicit  cost  of  implementing  an  intervention.  For  each  aspirational  goal,  we  first  calculated 
the  benefit-cost  index  and  used  this  information  to  rank  the  aspirational  research  goals  in  a 
way  that  accounted  for  effectiveness  and  cost.  We  then  compared  the  result  to  the  rankings 
of  “importance”  derived  from  the  RAND  ExpertLens  panel.  Using  the  index,  we  provide  a 
preliminary  snapshot  of  research  priorities,  with  the  provider  training  and  help-seeking  goals 
having  the  highest  benefit-cost  index  values  (93.3  and  49.6,  respectively);  the  values  for  the 
other  goals  drop  quickly,  starting  with  1.80  for  reduced  access  and  going  down  to  0.56  for  pre¬ 
diction.  Further  analysis  suggests  that  the  ordinal  ranking  of  goals  that  ranked  in  the  middle 
of  the  group  in  the  ExpertLens  exercise  are  sensitive  to  the  assumptions  used  to  parameterize 
the  model  while  the  goals  ranking  very  high  and  very  low  are  least  sensitive. 

We  also  took  the  index  value  rankings  and  plotted  them  against  the  cultural  acceptability 
rankings  to  highlight  the  strategies  that  look  very  promising  based  on  the  explicit  benefit-cost 
index  but  carry  high  implicit  implementation  costs  (i.e.,  those  that  ranked  low  with  respect 
to  cultural  acceptability)  and  that,  as  a  consequence,  would  need  to  be  discounted.  The  scat- 
terplot  in  Figure  S.l  shows  how  the  goals  align,  revealing  that  psychosocial  interventions,  help¬ 
seeking,  and  reduced  access  may  need  to  be  discounted. 

Drawing  on  recent  insights  from  the  economics  literature  on  research  and  development 
portfolio  choice  and  learning  value,  we  used  both  the  benefit-cost  index  and  future  learning 
potential  to  provide  even  more  insight  into  optimizing  a  suicide  prevention  research  portfolio. 
Figure  S.2  shows  how  the  12  strategies  rank  in  terms  of  perceived  learning  values,  with  the 

Figure  S.1 

Cultural  Acceptability  Ratings 


RAND  RR559-S.1 
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Figure  S.2 

Learning  Potential  Ratings 


RAND  RR5S9-S.2 


longer  error  bars  reflecting  more  potential  learning  value.  Although  provider  training  ranked 
first  in  terms  of  our  benefit-cost  index,  research  in  this  field  is  perceived  to  be  mature  and  thus 
has  a  low  learning  potential.  Help-seeking  ranked  slightly  lower  but  has  greater  learning  poten¬ 
tial.  Because  it  has  a  higher  learning  value  (i.e.,  greater  uncertainty),  it  is  possible  that  the  maxi¬ 
mum  potential  return  for  help-seeking  is  greater  than  the  maximum  potential  return  for  pro¬ 
vider  training.  The  figure  reflects  this  difference  in  that  the  top  of  the  error  bar  for  help-seeking 
is  higher  than  the  top  of  the  error  bar  for  provider  training  Gatekeeper  training,  which  also  has 
a  high  learning  potential,  similarly  offers  a  high  maximum  potential  return.  Population-based 
risk  reduction  is  another  strategy  with  a  high  learning  potential,  and  its  error  bar  overlaps  those 
of  several  adjacent  strategies  with  lower  learning  values. 

Determining  the  proper  investment  model  is  especially  complicated  because  of  the  inher¬ 
ent  lack  of  traditional  data  sets  to  support  the  empirical  analysis  of  research  outcomes.  We 
recognize  that  there  are  many  “unknowns,”  and  our  methodology,  combined  with  expert  elici¬ 
tation,  is  designed  to  organize  the  unknowns  into  a  more  manageable  problem.  Our  primary 
goal  was  to  provide  and  populate  a  metric  in  a  field  in  which  such  a  metric  is  lacking  and  to  do 
so  transparently.  We  believe  that  this  process,  and  the  ExpertLens  elicitation  exercise,  provides 
a  solid  ground  from  which  to  make  strategic  recommendations  that  can  inform  the  develop¬ 
ment  of  a  comprehensive  research  program  for  suicide  prevention  in  DoD. 


How  Can  This  Research  Be  Translated  into  Practice? 

Although  the  ultimate  objective  of  any  suicide  prevention  research  study  is  to  provide  empiri¬ 
cally  supported  insight  that  can  be  used  to  improve  or  inform  suicide  prevention  strategies. 
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there  is  a  “research-practice  gap”  when  it  comes  to  disseminating  research  conducted  in  aca¬ 
demic  settings  to  individuals  who  may  actually  henefit  from  the  hndings.  Based  on  a  literature 
review,  we  outlined  ten  components  for  successfully  adopting  research  programs  in  practice. 
We  augmented  this  information  hy  holding  discussions  with  experts  at  RAND  in  the  areas  of 
education,  substance  abuse,  occupational  safety,  criminal  justice,  behavioral  health,  and  health 
care  to  identify  “case  studies”  of  research  programs  that  have  been  successfully  implemented 
into  practice. 

The  first  three  components  focus  on  demonstrating  that  the  program  is  needed,  that  it  has 
the  potential  to  be  successful  at  a  particular  site,  and  that  it  is  rooted  in  scientific  evidence.  Suc¬ 
cessful  diffusion  begins  with  (1)  establishing  evidence  of  an  identified  need — that  is,  collecting 
data  to  demonstrate  to  organizational  leaders  and  adopters  on  the  ground  that  the  innovation 
has  relative  advantages  over  an  existing  program.  The  next  two  components  relate  to  examin¬ 
ing  the  supporting  evidence  for  a  new  program.  Ensuring  that  the  program  has  (2)  evidence  of 
research  quality  and  (3)  evidence  of  real-world  effectiveness  can  help  inform  leadership  decisions 
about  which  programs  to  adopt. 

Following  these  first  essential  components  are  those  related  to  having  internal  leadership 
and  financial  support  for  a  program.  These  components  are  (4)  leadership  buy-in  and  support 
from  key  stakeholders,  which  can  ensure  that  programs  are  championed  and  commissioned  by 
individuals  in  a  position  to  execute  change  within  an  organization  (which  is  essential  to  the 
other  nine  components).  Most  programs  also  cannot  exist  without  some  form  of  (5)  funding 
or  other  institutional  support  to  finance  the  program’s  start-up,  staff  pay,  and  the  infrastructure 
necessary  to  keep  a  program  running  on  a  day-to-day  basis. 

The  remaining  components  deal  primarily  with  the  process  of  garnering  support  and  dis¬ 
seminating  the  new  program  at  the  ground  level  (i.e.,  working  with  those  involved  in  the  actual 
implementation  of  the  program).  This  can  begin  with  (6)  collaboration  with  credible  sponsors, 
designing  research  programs  with  target  audiences  involved  at  the  development  stage.  This  also 
includes  collaboration  across  and  within  organizations — across  departments  and  disciplines — 
to  promote  innovation.  When  implementing  a  new  program  in  a  system  with  individuals  who 
may  be  used  to  or  invested  in  the  current  approach  or  who  are  now  expected  to  perform  addi¬ 
tional  or  different  duties,  the  (7)  provision  of  incentives  or  development  of  policies  can  be  helpful 
in  promoting  widespread  diffusion  of  the  program.  In  addition,  the  role  of  (8)  peer  networks 
supportive  of  adoption  cannot  be  underestimated,  because  individuals  learn  about  new  practices 
from  their  peers.  Providing  adopters  on  the  ground  with  (9)  dissemination  materials  can  facili¬ 
tate  the  process  of  diffusion  through  marketing  and  the  promotion  of  ideas.  Step-by-step  tool¬ 
kits  or  guidelines  can  assist  in  this  effort.  Finally,  when  a  new  program  is  adopted,  there  should 
be  an  (10)  expectation  of  a  cultural  shift  t\\3X  takes  time  and  will  require  the  continued  support 
of  new  program  adopters.  Awareness  of,  and  preparation  for,  resistance  and  delays  in  adoption 
can  help  leaders  remain  flexible  in  meeting  the  needs  of  their  organization  while  supporting 
those  charged  with  implementing  the  new  program. 


Recommendations 

In  light  of  the  study  results,  we  offer  a  series  of  ten  recommendations  (see  Table  S.2).  The  first 
is  an  overarching  recommendation,  and  the  remaining  nine  fall  into  three  general  categories: 
(1)  areas  in  which  DoD  should  prioritize  research  funding,  (2)  processes  that  DoD  should 
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adopt  or  enhance  to  more  efficiently  allocate  research  funding,  and  (3)  processes  that  DoD 
should  adopt  or  enhance  to  ensure  that  evidence-supported  suicide  prevention  strategies  are 
integrated  into  current  operations. 

Table  S.2 

Study  Recommendations 


Category 

Recommendations 

Overarching 

Recommendation 

1.  Leadership  is  needed  to  provide  strategic  guidance  for  implementing  a  unified 
research  strategy. 

Areas  in  which  DoD  should 
prioritize  research  funding 

2.  Eliciting  the  opinions  of  relevant  stakeholders  can  inform  the  development  of 
DoD's  research  priorities. 

3.  Research  investment  is  needed  to  prioritize  strategies  with  low  benefit-cost 
values;  policy  changes  are  needed  to  make  already  high  benefit-cost  strategies 
more  culturally  acceptable. 

4.  Funding  agencies  in  DoD  should  make  a  proactive  effort  to  fund  effectiveness 
research,  in  which  interventions  that  prior  research  (funded  by  DoD  or  another 
entity)  has  deemed  efficacious  are  evaluated  for  their  effectiveness  in  the  military 
context. 

Processes  that  DoD  should 
adopt  or  enhance  to  more 
efficiently  allocate  research 
funding 

5.  DoD  should  have  a  central  repository  to  identify  and  track  the  research  it  is 
funding  on  suicide  prevention. 

6.  The  designated  leadership  agency  in  DoD  (per  recommendation  1)  should 
continually  reevaluate  its  research  priorities  in  light  of  new  research  findings,  new 
policies,  and  the  adoption  of  new  suicide  prevention  strategies. 

Processes  that  DoD  should 
adopt  or  enhance  to  ensure 
that  evidence-supported 

7.  DoD  should  encourage  both  formal  and  informal  collaboration  across  the  DoD 
entities  responsible  for  funding  and  implementing  suicide  prevention  programs 
and  strategies. 

8.  Agencies  that  fund  suicide  prevention  research  and  those  responsible  for 
implementing  suicide  prevention  programs  should  keep  abreast  of  new  research, 
bearing  of  mind  the  quality  of  different  studies.  Efficiencies  may  be  gained  by 
creating  a  centralized  clearinghouse  for  this  purpose,  perhaps  capitalizing  on 

suicide  prevention  strategies  existing  sources 


are  integrated  into  current 
operations 

9.  Agencies  and  organizations  within  DoD  should  be  encouraged  to  adopt 
evidence-based  technologies.  Such  encouragement  may  include  funding, 
materials,  and  technical  assistance. 

10.  Both  leadership  buy-in  and  peer  engagement  are  key  in  promoting  new 
technologies. 
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Abbreviations 


AFRL 

Air  Eorce  Research  Eaboratory 

AFSP 

American  Eoundation  for  Suicide  Prevention 

AHRQ 
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U.S.  Department  of  Defense 
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Hawaii’s  Opportunity  Probation  with  Enforcement 

ICU 
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MOMRP 
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MSRC 

Military  Suicide  Research  Consortium 
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Operation  Iraqi  Freedom 
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post-traumatic  stress  disorder 

R&D 

research  and  development 

RCT 

randomized  controlled  trials 

RDT&E 

research,  development,  test,  and  evaluation 

SAMHSA 

Substance  Abuse  and  Mental  Health  Services  Administration 

STRONG  STAR 

South  Texas  Research  Organizational  Network  Guiding  Studies  on 
Trauma  and  Resilience 

TAGGS 

Tracking  Accountability  in  Government  Grants  System 

VA 

U.S.  Department  of  Veterans  Affairs 

VHA 

Veterans  Health  Administration 

CHAPTER  ONE 


Introduction 


Background 

The  U.S.  Department  of  Defense  (DoD)  has  been  increasingly  concerned  about  the  elevated 
rate  of  suicide  among  U.S.  service  members.  At  the  end  of  the  last  decade,  a  report  by  the 
RAND  Corporation  (Ramchand  et  ah,  2011)  and  the  final  report  of  the  congressionally  man¬ 
dated  Task  Force  on  the  Prevention  of  Suicide  by  Members  of  the  Armed  Forces  offered  a  series 
of  recommendations  to  help  strengthen  DoD’s  suicide  prevention  programs.  The  task  force’s 
final  recommendation  was  for  DoD  to  “create  a  unified,  strategic,  and  comprehensive  DoD 
plan  for  research  in  military  suicide  prevention  ensuring  that  the  DoD’s  military  suicide  pre¬ 
vention  research  portfolio  is  thoughtfully  planned  to  cover  topics  in  prevention,  intervention, 
and  postvention”  (U.S.  Department  of  Defense  Task  Force  on  the  Prevention  of  Suicide  by 
Members  of  the  Armed  Forces,  2010,  p.  ES-18). 

Shortly  after  the  task  force  report’s  release,  the  National  Action  Alliance  for  Suicide  Pre¬ 
vention  (NAASP),  a  public-private  partnership  charged  with  advancing  the  National  Strategy 
for  Suicide  Prevention,  established  a  Research  Prioritization  Task  Force  to  develop  “an  agenda 
for  research  with  the  stated  goal  to  reduce  morbidity  (attempts)  and  mortality  (deaths),  each  by 
at  least  20%  in  five  years  and  40%  or  greater  in  10  years,  if  implemented  fully  and  successfully” 
(NAASP,  2014,  p.  7;  emphasis  in  original).  This  report,  produced  by  the  RAND  National 
Defense  Research  Institute,  represents  an  effort  to  assist  DoD  in  creating  a  strategic  research 
plan  that  aligns  with  the  national  research  agenda. 


Study  Purpose 

This  study’s  objective  was  to  provide  guidance  that  DoD  could  use  to  develop  the  recom¬ 
mended  “unified,  strategic,  and  comprehensive  DoD  plan  for  research  in  military  suicide  pre¬ 
vention.”  To  meet  this  objective,  the  study  was  organized  around  three  overarching  research 
aims: 

1.  Catalog  current  research  being  conducted  on  suicide  prevention  of  direct  relevance  to 
military  personnel. 

2.  Examine  whether  current  research  maps  onto  DoD’s  strategic  research  needs  related  to 
suicide  prevention. 

3.  Ensure  that  any  proposed  DoD  research  strategy  aligns  with  the  national  research  strat¬ 
egy  and  is  integrated  with  DoD’s  data,  surveillance,  and  program  evaluation  strategies. 
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Approach 

We  took  a  multidisciplinary  approach  to  meeting  these  three  aims,  drawing  from  the  dis¬ 
ciplines  of  psychology,  epidemiology,  statistics,  and  economics.  We  began  hy  conducting  a 
comprehensive  scan  to  identify  who  in  the  United  States  was  funding  research  on  suicide  pre¬ 
vention  that  is  relevant  to  military  personnel  and  to  catalog  exactly  what  they  were  funding. 
Simultaneously,  we  assessed  DoD’s  suicide  prevention  research  needs.  We  did  so  hy  conducting 
an  online  consensus-huilding  exercise  with  DoD  representatives  who,  because  of  their  organi¬ 
zational  affiliation,  play  a  role  in  DoD’s  suicide  prevention  activities.  This  process  enabled  us  to 
rank  or  prioritize  DoD-specihc  research  goals,  rather  than  applying  national  rankings  gener¬ 
ated  by  the  NAASP. 

Our  next  task  was  to  compare  the  results  from  our  catalog  of  ongoing  research  to  the 
identified  needs  of  military  personnel.  This  mapping  exercise  provided  a  fairly  crude  under¬ 
standing  of  which  DoD-specific  research  needs  are  being  addressed  by  existing  research  and 
where  there  are  gaps.  We  improved  on  this  task  by  utilizing  a  benefit-cost  framework  captur¬ 
ing  the  input  from  the  needs  assessment  and  incorporating  additional  parameters  to  provide 
guidance  on  how  to  rank  or  prioritize  research  in  a  way  that  yields  maximum  impact  in  terms 
of  reduced  mortality. 

Part  of  a  unified  and  comprehensive  research  strategy  is  having  a  process  in  place  by 
which  research  findings  can  be  integrated  into  policy  and  practice  to  better  prevent  suicides. 
Thus,  our  next  task  was  to  identify  state-of-the  art  strategies  that  facilitate  the  diffusion  of 
research  into  practice.  We  did  so  primarily  through  a  literature  review,  which  was  enhanced  by 
case  studies  of  effective  dissemination  identified  in  both  the  literature  and  through  our  discus¬ 
sions  with  subject-matter  experts  at  RAND. 

The  final  task  was  to  synthesize  the  findings  from  the  aforementioned  tasks,  recogniz¬ 
ing  the  limitations  of  each,  to  construct  a  series  of  recommendations  across  three  domains: 
(1)  areas  in  which  DoD  should  prioritize  research  funding,  (2)  processes  that  DoD  should 
adopt  or  enhance  to  more  efficiently  allocate  research  funding,  and  (3)  processes  that  DoD 
should  adopt  or  enhance  to  ensure  that  evidence-supported  suicide  prevention  strategies  are 
integrated  into  current  operations. 


Organization  of  This  Report 

This  remainder  of  this  report  is  organized  into  six  chapters.  In  Chapter  Two,  we  present  a  sum¬ 
mary  of  all  research  related  to  suicide  prevention  that  is  directly  relevant  to  military  personnel 
and  funded  by  DoD,  the  U.S.  Department  of  Veterans  Affairs  (VA),  other  government  agen¬ 
cies,  and  private  foundations.  (Appendix  A  provides  descriptions  of  each  of  the  studies  that  met 
our  inclusion  criteria.)  In  Chapter  Three,  we  present  our  method  for  documenting  and  priori¬ 
tizing  DoD’s  suicide  prevention  research  needs  and  the  results  of  our  analysis.  In  Chapter  Four, 
we  describe  how  the  needs  identified  in  Chapter  Three  align  with  current,  ongoing  research 
catalogued  in  Chapter  Two.  In  Chapter  Five,  we  extend  our  analysis  of  research  prioritization 
by  constructing  an  empirical  model  to  estimate  how  strategic  investment  in  research  might 
quickly  and  efficiently  reduce  suicide  deaths.  It  is  becoming  increasingly  clear  both  inside 
and  outside  DoD  that  strategies  and  processes  are  needed  to  encourage  the  dissemination  of 
research  findings  and  incorporate  them  in  practice.  Thus,  in  Chapter  Six,  we  highlight  ten 


Introduction  3 


guiding  principles  that  need  to  be  considered  when  translating  research  findings  into  practice. 
Finally,  in  Chapter  Seven,  we  synthesize  the  material  from  the  preceding  chapters  and  present 
our  conclusions  and  recommendations  for  creating  “a  unified,  strategic,  and  comprehensive 
DoD  plan  for  research  in  military  suicide  prevention.” 

The  report  also  includes  a  series  of  appendixes.  The  tables  in  Appendix  A  present  key 
information  on  ongoing  studies  of  relevance  to  suicide  prevention  among  military  personnel. 
Appendix  B  provides  an  alternative  way  of  describing  the  allocation  of  research  funds  than 
what  is  presented  in  Chapter  Two.  Appendix  C  discusses  the  statistical  procedure  we  used  to 
extract  rankings  from  the  RAND  ExpertLens™  panel.  Appendix  D  shares  user  feedback  on 
RAND  ExpertLens.  Appendix  E  offers  a  detailed  description  of  the  sensitivity  analysis  that 
accompanies  the  empirical  model  presented  in  Chapter  Five.  Finally,  Appendix  F  discusses 
research  domains  and  approaches  for  assessing  research  quality. 


CHAPTER  TWO 


Current  Suicide  Prevention  Research  in  the  United  States  That  Is 
Directly  Relevant  to  Military  Personnel 


In  this  chapter,  we  catalog  current  research  being  conducted  in  the  United  States  on  suicide 
prevention  that  is  directly  relevant  to  military  personnel.  Cataloging  the  research  involved 
defining  the  hounds  of  research  abstraction;  creating  a  data  abstraction  form,  including  key 
categories  of  information  on  the  studies;  identifying  funders  of  suicide  prevention  research 
through  online  searches  and  by  asking  key  informants;  abstracting  the  research  using  online 
sources,  along  with  emails  and  interviews  with  researchers  and  research  portfolio  managers; 
and  systematically  categorizing  the  abstracted  research  to  quantify  the  areas  in  which  research 
is  being  conducted. 

The  following  references  provide  reviews  of  past  research  that  has  been  conducted  on  sui¬ 
cide  prevention  and  found  to  be  directly  relevant  to  military  personnel: 

•  Rajeev  Ramchand,  Joie  Acosta,  Rachel  M.  Burns,  Lisa  H.  Jaycox,  and  Christopher  G. 
Pernin,  The  War  Within:  Preventing  Suicide  in  the  US.  Military,  Santa  Monica,  Calif: 
RAND  Corporation,  MG-953-OSD,  2011. 

•  Committee  on  Gulf  War  and  Health,  Gulf  War  and  Health,  Vol.  6:  Physiologic,  Psycho¬ 
logic,  and  Psychosocial  Effects  of  Deployment-Related  Stress,  Washington,  D.C.:  National 
Academies  Press,  2008. 


Methods 

Inclusion  and  Exclusion  Criteria 

Before  beginning  the  task  of  abstraction,  we  developed  a  set  of  criteria  and  corresponding 
definitions,  which  we  used  to  determine  which  studies  should  be  included  and  which  should 
be  excluded.  We  applied  the  inclusion  and  exclusion  criteria  sequentially,  such  that  the  study 
first  needed  to  be  considered  “recent  research”  to  be  included,  and  then  the  other  criteria  were 
applied.  Working  with  the  Defense  Suicide  Prevention  Office,  we  established  the  following 
definitions  for  inclusion: 

•  Recent  is  any  project  funded  in  or  after  the  year  2005. 

•  Research  is  any  study  that  advances  knowledge  and  theory,  not  evaluations  that  focus  on 
the  implementation  of  a  specific  program  and  are  limited  in  scope.  We  did  include  large- 
scale  program  evaluations  that  seek  to  produce  generalizable  knowledge  that  goes  beyond 
the  specific  program. 


5 


6  Developing  a  Research  Strategy  for  Suicide  Prevention  in  the  Department  of  Defense 


•  Direct  relevance  to  military  personnel  means  that  the  research  examines  military  or  recent 
veteran  samples  or  nonclinical  samples  of  civilians  comparable  in  age  to  military  person¬ 
nel.  We  operationalized  this  concept  hy  including  studies  in  which  the  sample  or  focus 
was  on  a  population  of  a  mean  age  between  13  and  65  years. 

We  excluded  studies  from  our  abstraction  if  the  aggregate  funding  occurred  before  2005, 
the  study  could  not  be  considered  as  “research,”  or  the  study  was  not  directly  relevant  to  mili¬ 
tary  personnel.  Furthermore,  research  was  excluded  if  the  focus  was  on  suicide  or  suicide  pre¬ 
vention  among  individuals  with  serious  mental  illness,  such  as  schizophrenia  or  bipolar  disor¬ 
der.  Table  2.1  provides  an  overview  of  the  inclusion  and  exclusion  criteria. 

Identifying  Studies  for  Abstraction 

Through  a  series  of  interviews  and  a  review  of  past  RAND  work  on  suicide  prevention  research, 
we  sought  to  identify  all  studies  with  direct  relevance  to  military  personnel  that  were  funded 
by  the  U.S.  Department  of  Health  and  Human  Services  (DHHS),  VA,  and  DoD.  In  addi¬ 
tion,  we  sought  to  identify  all  studies  meeting  our  inclusion  criteria  funded  by  the  Ameri¬ 
can  Foundation  for  Suicide  Prevention  (AFSP).  Our  focus  on  these  agencies  and  organiza¬ 
tions  suggests  that  we  may  be  missing  studies  funded  by  other  government  agencies  (e.g.,  the 
U.S.  Department  of  Homeland  Security,  U.S.  Department  of  Education),  private  foundations 
(e.g.,  the  MacArthur  Foundation),  or  private  companies  (e.g.,  pharmaceutical  or  medical  device 
manufacturers). 

U.S.  Department  of  Health  and  Human  Services 

We  used  the  Tracking  Accountability  in  Government  Grants  System  (TAGGS)  to  identify 
research  funded  by  DHHS  (see  TAGGS,  undated).  TAGGS  is  the  reporting  tool  for  data  on 
all  grants  awarded  by  the  11  operating  divisions  of  DHHS.  We  conducted  TAGGS  searches 
between  the  months  of  September  and  December  2012.  Using  the  advanced  search  in  the 
system,  we  set  the  key  abstract  word  as  “suicide.”  Gonsistent  with  our  inclusion  and  exclusion 
criteria,  we  set  the  “fiscal  year  award  made”  as  2005  to  2012  and  focused  on  awards  categorized 
as  “evaluation,”  “scientific/health  research,”  or  “social  science  research.” 


Table  2.1 

Inclusion  and  Exclusion  Criteria  for  Suicide  Prevention  Research  Abstraction 


Inclusion  Criteria 


Exclusion  Criteria 


1.  Recent:  Studies  funded  in  or  after  2005. 

2.  Research:  Studies  that  advance  knowledge 
and  theory,  not  evaluations  that  focus  on  the 
implementation  of  a  specific  program  and  are  limited 
in  scope.  We  included  large-scale  program  evaluations 
that  seek  to  produce  generalizable  knowledge  that 
goes  beyond  the  specific  program  as  "research." 

3.  Direct  relevance  to  military  personnel:  Studies  that 
could  be  applicable  to  military  personnel  in  that  the 
research  examines  military  or  recent  veteran  samples 
or  nonclinical  samples  of  civilians  comparable  in  age  to 
military  personnel  (i.e.,  adolescent,  adult). 


Program  evaluations  that  do  not  produce  generalizable 
knowledge 

Child  sample,  operationalized  as  child  focus  or  mean 
age  under  13 

Older  adult  sample,  operationalized  as  mean  age 
over  65 

Serious  mental  illness  (e.g.,  schizophrenia  or  other 
psychotic  disorder,  bipolar  disorder)  or  borderline 
personality  disorder 
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U.S.  Department  of  Veterans  Affairs 

To  identify  suicide  prevention  research  funded  by  the  VA,  we  used  the  Research  Studies  and 
Implementation  Projects  search  engine  on  the  website  of  the  VA’s  Office  of  Health  Services 
Research  and  Development  (see  HSR&D,  undated).  We  conducted  our  search  in  November 
2012  using  “suicide”  as  the  keyword. 

We  found  the  VA  search  engine  to  have  some  limitations.  For  example,  funding  amounts 
are  not  provided  with  the  search  results,  and  the  database  does  not  contain  many  studies 
funded  before  2009.  For  the  missing  information  and  earlier  studies,  we  sought  assistance 
from  the  VA  directly.  In  June  2013,  the  Office  of  Health  Services  Research  and  Development 
provided  a  list  of  suicide  prevention  research  funded  by  the  VA  beginning  in  fiscal  year  2006. 

American  Foundation  for  Suicide  Prevention 

The  AFSP  is  the  leading  nonprofit  organization  for  funding  suicide  prevention  research.  It 
awards  research  grants  to  established  social  science  researchers,  medical  professionals,  and 
young  scholars  in  pursuit  of  an  advanced  degree.  The  research  grant  awards  are  listed  by  the 
year  awarded  on  AFSP’s  website,  which  also  includes  general  information  on  each  study  (see 
AFSP,  undated).  Because  the  organization’s  aim  is  to  fund  research  pertaining  to  suicide  pre¬ 
vention,  we  chose  not  to  narrow  our  review  and  assessed  all  the  research  grants  that  were  pub¬ 
licly  listed  when  we  conducted  our  review  in  November  and  December  2012. 

U.S.  Department  of  Defense 

Identifying  and  abstracting  research  funded  by  DoD  poses  unique  challenges.  As  a  large,  com¬ 
plex  federal  department,  DoD  comprises  several  components,  organizations,  and  agencies,  any 
one  of  which  may  support  research  activities  related  to  suicide  prevention.  Unlike  in  DHHS, 
which  is  also  diverse,  there  is  no  readily  available  central  repository  for  cataloguing  research 
activities  funded  across  DoD.  For  example,  each  of  the  service  components  (the  Army,  Air 
Force,  Navy,  and  Marine  Corps)  support  intramural  and  extramural  research  activities  related 
to  suicide  prevention,  either  through  their  operations  and  maintenance  (O&M)  funding  or 
through  their  research,  development,  test,  and  evaluation  (RDT&E)  funding.  The  specific 
funding  stream  used  to  support  the  research  can  be  difficult  to  identify.  Studies  funded  by 
specific  RDT&E  programs  or  through  designated  research  management  program  offices  are 
more  readily  identifiable. 

One  of  the  primary  funding  streams  for  research  on  suicide  prevention  within  DoD 
is  the  Congressionally  Directed  Medical  Research  Programs  (CDMRP).  The  CDMRP  was 
established  by  Congress  in  1992  to  oversee  and  administer  medical  research  on  specific 
disease-related  topics  designated  by  Congress.  It  has  a  specific  disease  focus  and  involves  con¬ 
sumers  in  the  process  (from  setting  priorities  to  reviewing  proposals).  Congress  designated  the 
U.S.  Army  as  executive  agent  of  this  program,  and  while  the  funded  research  is  not  required 
to  be  directly  relevant  to  military  populations  or  settings,  many  of  the  studies  have  a  specific 
focus  on  the  military.  Among  the  most  widely  recognized  programs  managed  by  the  CDMRP 
are  its  breast  cancer  and  prostate  cancer  programs.  The  CDMRP  is  managed  by  the  U.S. 
Army  Medical  Research  and  Materiel  Command;  its  funding  is  allocated  through  specific  line 
items  set  by  Congress  (not  the  President)  in  the  Defense  Health  Budget  as  part  of  the  yearly 
national  defense  appropriations  legislation.  While  DoD  can  request  funding  for  research  on 
similar  topics  through  its  own  budget  requests  to  the  President,  the  Secretary  of  Defense,  and 
other  defense  leaders,  it  may  not  use  congressionally  set  appropriations  for  the  CDMRP  for 
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Other  reasons.  In  2007,  Congress  initiated  the  Psychological  Health  and  Traumatic  Brain  Injury 
Research  Program  through  CDMRP,  which  also  provided  a  new  research  funding  stream  to 
support  suicide  prevention  research. 

U.S.  Army  Medical  Research  and  Materiel  Command  also  manages  several  other  research 
programs,  priorities,  and  funding  lines,  which  are  set  through  the  President’s  Budget.  The 
Military  Operational  Medicine  Research  Program  (MOMRP)  has  specific  areas  of  research 
related  to  suicide  prevention.  Several  years  ago,  MOMRP  took  on  an  effort  to  “integrate  and 
synchronize  U.S.  Department  of  Defense  and  civilian  efforts  to  implement  a  multidisciplinary 
research  approach  to  suicide  prevention”  and  funded  the  Military  Suicide  Research  Consor¬ 
tium  (MSRC)  through  a  large  grant  to  Florida  State  University  and  the  Denver  VA  Medical 
Center  (see  MSRC,  undated). 

Identifying  funded  research  activities  through  CDMRP  and  MOMRP  was  relatively 
straightforward,  given  their  emphasis  and  management.  CDMRP  has  a  searchable  online 
mechanism  for  identifying  funded  awards.  MOMRP  staff  also  maintain  detailed  informa¬ 
tion  on  all  funded  research  studies.  However,  because  these  are  not  the  only  DoD  entities 
supporting  research  on  suicide  prevention,  we  also  had  to  rely  on  key  contacts  in  various  DoD 
organizations  and  agencies,  as  well  as  other  knowledgeable  contacts,  for  assistance  in  identify¬ 
ing  relevant  research.  In  this  way,  we  gathered  information  on  DoD-funded  studies  through 
a  combination  of  online  resources  and  communication  with  research  portfolio  managers  and 
funded  investigators  across  DoD. 

Data  Abstraction 

We  developed  a  custom  abstraction  tool  to  collect  pertinent  data  on  studies  that  matched  our 
criteria.  Three  members  of  the  RAND  study  team  abstracted  all  the  information. 

At  times,  a  public  search  engine  or  databases  did  not  provide  all  the  information  needed 
for  abstraction.  We  filled  such  gaps  through  additional  online  research  or  direct  communica¬ 
tion  with  the  study’s  principal  investigator  or  a  representative  from  the  funding  agency.  Still, 
some  detailed  information  remained  unavailable,  in  which  case  we  left  the  field  blank. 

Using  the  available  information,  we  categorized  each  study  according  to  the  categories 
established  by  the  NAASP  and  the  MOMRP  Continuum  of  Care  categories.  The  NAASP 
categories  were  originally  developed  based  on  surveys  with  suicide  prevention  stakeholders, 
including  individuals  with  personal  experience  with  suicide,  health  care  providers,  policy¬ 
makers  and  administrators,  and  researchers  who  identified  aspirational  goals  for  suicide  pre¬ 
vention  research  (NAASP,  undated).  We  categorized  recent  suicide  prevention  research  into 
one  or  more  of  the  NAASP  categories:  prevention  of  reattempts;  enhanced  continuity  of 
care;  provider  and  gatekeeper  training;  affordable,  accessible,  and  effective  care;  psychoso¬ 
cial  interventions  for  those  at  risk;  risk  and  protective  factor  interactions;  stigma  reduction; 
population-based  risk  reduction/resilience-building;  prediction  of  imminent  risk;  improved 
biological  interventions;  reduction  in  access  to  lethal  means;  and  population-based  screening. 

MOMRP  developed  a  continuum  of  care  framework  to  describe  military  suicide  research 
needs  (Gutierrez,  Joiner,  and  Castro,  2012).  We  also  categorized  recent  suicide  prevention 
research  into  one  or  more  of  the  MOMRP  categories:  prevention  education  and  training,  early 
screening/intervention,  assessment,  treatment,  recovery  and  return  to  duty,  postvention,  and 
epidemiology  and/or  basic  science/neurological  mechanisms.  (Note  that  the  first  six  categories 
are  part  of  the  continuum,  whereas  the  last  category  is  part  of  the  foundation  for  the  other 
research  categories  in  the  MOMRP  approach  to  suicide  research  classification.) 
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We  matched  the  appropriate  NAASP  and  MOMRP  categories  to  the  studies  hy  using 
all  available  information  on  each  study’s  methods,  objectives,  and  conclusions.  The  aims  of 
the  research  dictated  its  categorization.  For  example,  a  study  aimed  at  examining  effective 
treatment  techniques  would  likely  be  categorized  as  fitting  under  the  “affordable,  accessible, 
and  effective  care”  NAASP  category  and  the  “treatment”  MOMRP  category.  Many  studies 
explored  multiple  aims  and  therefore  were  placed  in  multiple  categories.  The  “risk  and  protec¬ 
tive  factor  interactions”  NAASP  category  and  the  MOMRP  category  “epidemiology  and/or 
basic  science/neurological  mechanisms”  were  used  to  identify  studies  that  examined  the  factors 
that  contribute  to  suicide  risk  that  could  be  applied  to  prevention.  The  MOMRP  categories 
“early  screening/intervention”  and  “assessment”  differ  in  their  application  to  a  potential  act 
of  suicide.  Whereas  a  survey  that  screens  all  new  Army  recruits  would  fit  within  the  “early 
screening/intervention”  category,  a  study  that  validates  a  checklist  evaluating  suicidal  ideation 
would  be  considered  “assessment.”  Studies  that  were  categorized  as  “assessment”  were  often 
also  included  in  the  NAASP  category  “prediction  of  imminent  risk.”  Understandably,  as  they 
both  specifically  involve  those  who  have  attempted  suicide,  the  MOMRP  category  “recov¬ 
ery  and  return  to  duty”  and  the  NAASP  category  “prevention  of  reattempts”  often  appeared 
together.  Additional  information  on  the  studies’  methods  also  informed  their  assignment  to 
specific  categories — for  example,  if  information  on  a  study’s  sample  was  relevant  to  the  “pro¬ 
vider  and  gatekeeper  training”  NAASP  category. 

All  ratings  were  predominately  completed  by  two  raters,  who  alerted  a  third  rater  to  any 
studies  for  which  the  categorization  was  ambiguous.  For  these  ambiguous  cases,  a  consensus 
rating  method  was  used.  A  third  rater  also  checked  the  assessments  of  the  two  main  raters  to 
ensure  that  they  were  consistent. 

While  investigating  studies  for  categorization,  the  raters  additionally  noted  the  composi¬ 
tion  of  the  study’s  participants.  Raters  noted  whether  the  study  participants  were  civilians,  vet¬ 
erans,  or  members  of  the  armed  services.  If  the  latter,  they  noted  the  specific  service  or  services 
(e.g..  Army,  Navy).  Table  2.2  shows  the  information  we  gathered  from  the  studies  that  satisfied 
the  inclusion  criteria  presented  in  Table  2.1  and  the  NAASP  and  MOMRP  categories. 


Results 

This  section  is  divided  into  four  categories.  First,  we  present  a  summary  of  our  search  results, 
highlighting  in  aggregate  the  studies  we  identified  and  the  number  and  amount  funded  by 
and  across  the  four  sources  (DHHS,  VA,  AFSP,  and  DoD).  Then,  we  present  the  number  and 
amount  funded  by  source  across  each  of  the  NAASP  and  MOMRP  categories.  Though  not 
a  direct  measure,  the  number  of  studies  may  be  loosely  interpreted  as  the  number  of  different 
ideas  or  hypotheses  that  are  being  pursued.  However,  the  “depth”  of  each  study’s  investiga¬ 
tion  will  be  constrained  by  the  funds  available.  Thus,  we  also  present  the  amount  allocated  to 
each  research  category.  Finally,  we  present  information  on  the  number  and  amount  funded  by 
source  with  respect  to  study  participants. 

It  is  important  to  note  that,  in  our  analyses  across  the  NAASP  and  MOMRP  catego¬ 
ries,  the  total  funding  estimates  are  frequently  overestimates.  This  is  because  studies  could  be 
categorized  into  more  than  one  area,  and  it  is  impossible  to  tease  out  the  funds  spent  on  each 
objective.  Thus,  we  attributed  the  total  dollar  amount  for  each  study  to  a  given  category.  As 
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Table  2.2 

Data  Abstraction  Categories 


Study  Information  Collected 


Category 


Study/contract  number 
Funder 

Project/award  title 
Project  leader 
Organization  funded 
Project  start  date 
Project  end  date 
Total  funding 
Study  abstract 
Study  aims 

Resulting  publications 

Study  design 
Experimental 
Quasiexperi  mental 
Observational 
Case  study 
Sample  size 
Data  source 

Participants 
Air  Force 
Army 
Marines 
Navy 

Coast  Guard 

National  Guard/Reserve 

Veteran 

Civilian 


NAASP  category 

Prevention  of  reattempts 
Enhanced  continuity  of  care 
Provider  and  gatekeeper  training 
Affordable,  accessible,  and  effective  care 
Psychosocial  interventions  for  those  at  risk 
Risk  and  protective  factor  interactions 
Stigma  reduction 

Population-based  risk  reduction/resilience-building 
Prediction  of  imminent  risk 
Improved  biological  interventions 
Reduction  in  access  to  lethal  means 
Population-based  screening 

MOMRP  category 

Prevention  education  and  training 
Early  screening/intervention 
Assessment 
Treatment 

Recovery  and  return  to  duty 
Postvention 

Epidemiology  and/or  basic  science/neurological  mechanisms 


such,  the  totals  are  not  reliable  because  of  double- counting,'  but  the  information  is  still  useful 
for  examining  the  relative  amounts  of  funding  spent  on  different  topics. 

Studies  Identified,  Funders,  and  Funding 

The  majority  of  the  studies  we  reviewed  did  not  meet  our  established  criteria  for  inclusion.  We 
documented  reasons  for  exclusion  whenever  possible.  The  most  common  reason  for  exclusion 
differed  by  organization.  The  systematic  search  of  studies  funded  by  DHHS  yielded  numerous 
results  entirely  unrelated  to  suicide  prevention.  In  contrast,  most  studies  funded  by  the  AFSP 
were  relevant,  and  the  exclusions  were  predominantly  because  of  study  sample  characteristics. 
While  most  reasons  for  exclusion  were  derived  from  our  inclusion  and  exclusion  criteria,  we 
also  commonly  encountered  duplicate  entries  for  the  same  study  and  occasionally  encountered 
entries  that  were  not  actually  research  studies.  The  reasons  for  exclusion  are  summarized  in 
Table  2.3. 

Table  2.4  provides  an  overview  of  the  number  of  studies  reviewed  and  ultimately  included 
from  the  four  funders,  as  well  as  the  total  amount  of  funding  associated  with  these  studies.  Key 
information  on  all  the  studies  included  in  this  analysis  is  presented  in  Appendix  A. 


*  See  Appendix  B  for  an  alternative  allocation  strategy  in  which  we  divided  the  total  funding  per  grant  by  the  number  of 
areas  in  which  a  study  was  categorized,  with  each  category  receiving  an  equal  proportion. 
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Table  2.3 

Reasons  for  Exclusion 


Reason 

Number  of  Excluded  Studies 

No  relevance  (e.g.,  "suicide  cells"  in  biology) 

1,369 

Duplicates 

302 

Not  research 

63 

Serious  mental  illness  (e.g.,  schizophrenia) 

59 

Elderly  sample 

16 

Missing  substantial  information  (e.g.,  name,  abstract),  mistake 
entry,  nonworking  link 

9 

Small-child  sample 

2 

NOTE:  The  total  sample  size  was  1,820  studies.  Because  of  the  lack  of  a  unified  reporting  system 
for  most  DoD-funded  studies,  this  sample  includes  CDMRP  studies  only. 


U.S.  Department  of  Health  and  Human  Services 

With  respect  to  studies  funded  by  DHHS,  the  search  criteria  described  here  yielded 
1,542  results.  After  an  assessment  using  our  inclusion  and  exclusion  criteria,  we  incorporated 
49  of  the  results  into  our  abstraction  of  recent  research  on  suicide  prevention.  The  49  stud¬ 
ies  totaled  $43,215,850  in  funding.  Of  the  49  studies  meeting  the  inclusion  criteria,  44  were 
funded  by  the  National  Institutes  of  Health  (NIH),  which  accounted  for  $39,745,981,  or 
92  percent  of  relevant  DHHS  funding.  The  National  Center  for  Injury  Prevention  and  Control 
at  the  Centers  for  Disease  Control  and  Prevention  (CDC-NCIPC)  and  the  Agency  for  Health¬ 
care  Research  and  Quality  (AHRQ)  funded  three  and  two  of  the  included  studies,  respec¬ 
tively.  No  study  or  evaluation  by  the  Substance  Abuse  and  Mental  Health  Services  Adminis¬ 
tration  (SAMHSA)  met  our  inclusion  criteria. 

U.S.  Department  of  Veterans  Affairs 

Our  search  for  VA  studies  generated  30  results,  hve  of  which  met  our  inclusion  criteria.  The 
Office  of  Health  Services  Research  and  Development  provided  us  with  an  additional  ten  stud¬ 
ies,  for  a  total  of  15  included  studies  with  $8,125,622  in  associated  funding. 

American  Foundation  for  Suicide  Prevention 

The  AFSP  provided  information  on  108  studies;  we  abstracted  92  of  them,  representing 
$7,214,781  in  funding. 

U.S.  Department  of  Defense 

Since  its  establishment  in  2007,  the  CDMRP’s  Psychological  Health  and  Traumatic  Brain 
Injury  Research  Program  has  awarded  312  grants  (some  being  multiple  grants  to  the  same 
research  effort).  We  reviewed  the  312  grants  and  found  26  studies  that  fit  our  inclusion  criteria, 
representing  $48,859,439  in  funding. 

We  identified  33  other  relevant  DoD  studies;  combined  with  the  CDMRP  funding, 
this  brought  total  DoD  funding  for  suicide  prevention  research  to  $107,274,505.  (We  were 
unable  to  obtain  from  U.S.  Army  Medical  Research  and  Materiel  Command  the  total  fund¬ 
ing  amount  for  four  studies,  as  noted  in  Appendix  A.)  We  also  identified  two  studies — the 
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Table  2.4 

Funders  of  Suicide  Prevention  Research 


Organization 

Number  of  Studies 

Number  of  Studies 
Meeting  Inclusion  Criteria 

Total  Funding  ($) 

DHHS  (all) 

1,542 

49 

43,215,850 

NIH 

1,518 

44 

39,745,981 

CDC-NCIPC 

14 

3 

1,693,548 

AHRQ 

10 

2 

1,776,321 

AFSP 

108 

92 

7,214,781 

VA 

— 

15 

8,125,622 

DoD  (all) 

— 

61^ 

107,274,505^ 

Air  Force  Research  Laboratory 
(AFRL) 

— 

3 

10,325,000 

CDMRP 

— 

26 

48,859,439 

Defense  Centers  of  Excellence 
(DCoE) 

— 

5 

6,810,573 

U.S.  Department  of  the  Navy 

— 

1 

6,989,000 

MOMRP— MSRC 

— 

7 

3,737,269^ 

MOMRP — other 

— 

19 

30,553,224'= 

Army  STARRS  (DoD,  NIH) 

N/A 

1 

65,000,000 

Total 

1,680 

218 

230,830,758 

NOTE:  The  table  does  not  include  the  Millennium  Cohort  Study  or  STRONG  STAR  because  it  was  not  possible  to 
precisely  determine  the  proportion  of  funds  going  to  suicide  prevention  within  these  larger  studies. 

The  total  number  of  VA  and  DoD  studies  could  not  be  determined.  Unlike  the  review  of  other  sources  of 
funding,  there  was  no  specific  search  engine  or  consistent  protocol  to  identify  most  DoD  and  VA  studies.  These 
studies  were  largely  identified  through  personal  communication  with  funders  and  investigators,  as  well  as 
through  our  literature  review. 

^  Total  DoD  funding  does  not  include  Army  STARRS 

^  The  total  funding  amount  for  MOMRP — MSRC  reflects  only  the  four  of  seven  studies  with  available  funding 
information  (i.e.,  three  studies  were  missing  funding  information). 

The  total  funding  amount  for  MOMRP — other  reflects  only  the  17  studies  with  available  funding  information, 
(i.e.,  two  studies  were  missing  funding  information). 


Millennium  Cohort  study  and  the  South  Texas  Research  Organizational  Network  Guiding 
Studies  on  Trauma  and  Resilience  (STRONG  STAR)  multidisciplinary  post-traumatic  stress 
disorder  (PTSD)  research  consortium — that  are  related  to  hut  do  not  focus  exclusively  on  sui¬ 
cide.  Our  investigation  revealed  that  these  studies  have  dedicated  a  relatively  small  amount  of 
their  funding  directly  to  suicide  prevention,  and  we  could  not  accurately  quantify  this  amount. 
Thus,  these  studies  are  excluded  from  all  subsequent  analyses  in  this  report.  (See  Box  2.1  for  a 
description  of  each  study.) 

Aggregated  Results 

Across  all  funding  sources,  a  total  of  218  studies  met  our  inclusion  criteria,  with  $230,830,758 
in  total  funding. 
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Box  2.1 

Large-Scale  Studies  Funded  by  DoD 


Army  STARRS 

Army  STARRS  is  a  comprehensive,  collaborative  study  focused  on  suicide.  The  five-year  study  was  funded 
in  2009  in  response  to  the  growing  suicide  rate  in  the  armed  forces.  The  $65  million  in  funding  jointly 
provided  by  DoD  and  NIMH  supports  the  work  of  principal  investigators  from  the  Uniformed  Services 
University  of  the  Health  Sciences,  Harvard  Medical  School,  the  University  of  Michigan,  and  the  University 
of  California,  San  Diego.  Army  STARRS  is  organized  into  four  primary  research  components:  the  Historical 
Data  Study,  the  All  Army  Study,  the  New  Soldier  Study,  and  the  Soldier  Health  Outcomes  Study.  While 
varied  in  approach,  all  Army  STARRS  research  supports  the  project's  main  emphasis  of  protecting  the 
health  of  soldiers. 

Millennium  Cohort  Study 

The  Millennium  Cohort  Study  is  an  ongoing  DoD-funded  prospective  cohort  study  that  boasts  a  sample 
of  more  than  150,000  active  and  non-active  military  personnel  representing  all  service  branches.  Since 
2001,  researchers  have  administered  triennial  surveys  to  an  increasing  number  of  enrolled  participants. 
The  surveys  include  validated  instruments  for  self-assessing  physical  and  mental  functional  status.  Results 
obtained  from  the  self-assessments  can  be  linked  to  a  wide-ranging  supply  of  individual  information, 
such  as  the  subject's  occupation  or  past  health  care  utilization.  The  study  involves  continually  surveying 
all  subjects  every  three  years  until  2022,  for  a  21-year  period  of  data  collection.  In  addition  to  assessing 
service  members'  health  status  over  time,  the  Millennium  Cohort  Study  aims  to  serve  as  a  rich  data  source 
that  can  provide  strong  support  for  current  and  future  epidemiological  studies.  Preliminary  data  analysis 
has  already  been  published  in  medical  journals. 

STRONG  STAR  Multidisciplinary  PTSD  Research  Consortium 

STRONG  STAR  is  a  multi-institution  research  consortium  funded  by  DoD.  Specifically,  STRONG  STAR 
was  allocated  $35,989,697  from  fiscal  years  2007  to  2013,  which  was  distributed  among  projects  with 
differing  approaches  but  a  matching  objective:  developing  and  evaluating  effective  interventions  for  the 
"detection,  prevention,  and  treatment  of  combat-related  posttraumatic  stress  disorder  (PTSD)"  (STRONG 
STAR,  undated).  The  large  population  of  Operation  Iraqi  Freedom  (OIF)  and  Operation  Enduring  Freedom 
(OEF)  veterans  in  south-central  Texas  aids  in  the  inclusion  of  both  recently  discharged  veterans  and  active- 
duty  service  members. 


The  single  largest  study  was  the  Army  Study  to  Assess  Risk  and  Resilience  in  Service- 
members  (Army  STARRS),  for  which  $50  million  was  provided  by  the  U.S.  Army  and 
$15  million  was  provided  by  the  National  Institute  of  Mental  Health  (NIMH,  which  also 
administers  the  program),  for  a  total  of  $65  million  (see  Box  2.1).  Given  the  magnitude  of  this 
study,  we  include  it  in  our  estimate  and  in  Table  2.4  in  the  grand  total  being  spent  on  sui¬ 
cide  prevention  research.  However,  we  exclude  it  from  the  remainder  of  our  funding  analyses 
because  its  sheer  size  would  complicate  efforts  to  interpret  funding  trends. 

With  respect  to  the  total  number  of  studies  funded,  the  single  biggest  funder  was  the 
AFSP,  which  funded  92  studies  meeting  our  inclusion  criteria. 

With  respect  the  total  amount  of  funding,  the  biggest  single  funder  was  DoD,  which 
funded  a  total  of  $107,274,505  in  research  related  to  suicide  prevention,  accounting  for 
65  percent  of  all  research  funding  (see  Table  2.4  and  Figure  2.1),  excluding  the  Army  STARRS, 
Millennium  Cohort,  and  STRONG  STAR  studies.  As  shown  in  Figure  2.2,  almost  half  of 
these  funds  came  from  CDMRR  The  NIH  is  the  largest  non-DoD  contributor,  providing 
68  percent  of  the  $58,556,253  in  non-DoD  funds  going  toward  suicide  prevention  research 
(Figure  2.3). 

Funding,  by  NAASP  Category 
Number  of  Studies 

Across  all  funders,  the  most  common  area  of  recent  research  involved  psychosocial  interven¬ 
tions  for  those  at  risk  (78  studies;  see  Table  2.5).  The  next  most  common  areas  of  research 
were  studies  of  risk  and  protective  factor  interactions  (58  studies),  prediction  of  imminent  risk 
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Figure  2.1 

Proportion  of  Suicide  Prevention  Funding 
from  Different  Sources 


AFSP 


NOTE:  Total  funding  =  $165,830,758.  The  figure 
does  not  include  Army  STARRS,  the  Millennium 
Cohort  Study,  or  STRONG  STAR. 
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Figure  2.2 

Proportion  of  Suicide  Prevention  Funding  from 
Different  Sources:  DoD 


MOMRP- 

MSRC 


AFRL 


MOMRP-Other 


Department  of 
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DCoE 


NOTE:  Total  funding  =  $107,274,505.  The  figure  does  not 
include  Army  STARRS,  the  Millennium  Cohort  Study,  or 
STRONG  STAR. 
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Figure  2.3 

Proportion  of  Suicide  Prevention  Funding  from 
Different  Sources:  Non-DoD 


NOTE:  Total  funding  =  $58,556,253. 

RAND  RR559-2.3 


(48  studies),  and  affordable,  accessible,  and  effective  care  (45  studies).  A  moderate  amount  of 
research  is  being  conducted  on  improved  biological  interventions  (35  studies),  enhanced  conti¬ 
nuity  of  care  (27  studies),  and  population-based  risk  reduction/resilience-building  and  provider 
and  gatekeeper  training  (23  studies  in  each  category).  We  found  fewer  studies  on  population- 
based  screening  (11  studies),  the  prevention  of  reattempts  among  those  who  have  previously 
attempted  suicide  (11  studies),  and  stigma  reduction  (11  studies),  and  there  were  only  three 
studies  on  reducing  access  to  lethal  means.  ^ 

While  it  is  informative  to  review  the  proportion  of  each  funder’s  research  portfolio  that 
falls  in  each  NAASP  category  (Table  2.5),  another  way  to  look  at  these  data  is  to  consider  the 
proportion  of  research  within  each  NAASP  category  that  is  being  funded  by  each  of  the  major 
funders  (Figure  2.4).  As  shown,  DoD  is  funding  the  majority  of  research  on  stigma  reduc¬ 
tion  (64  percent  of  all  studies  in  this  category)  and  affordable,  accessible,  and  effective  care 
(67  percent);  it  is  funding  much  of  the  research  on  improved  biological  interventions  (9  per¬ 
cent),  prediction  of  imminent  risk  (13  percent),  or  prevention  of  reattempts  (18  percent).  These 
latter  areas  of  research  are  being  driven  largely  by  funders  outside  DoD.  Notably,  the  AFSP  is 
funding  three-quarters  of  all  current  studies  on  improved  biological  interventions,  DHHS 
is  funding  more  than  half  of  studies  on  preventing  reattempts,  and,  though  the  funding  is  rela¬ 
tively  dispersed,  the  VA  is  supporting  one-quarter  of  all  studies  on  population-based  screening. 


^  The  limited  amount  of  research  on  means  restriction  is  partly  the  result  of  a  1996  congressional  directive  that  “none  of 
the  funds  made  available  for  injury  prevention  and  control  at  the  Centers  for  Disease  Control  and  Prevention  may  be  used 
to  advocate  or  promote  gun  control”  (Kellermann  and  Rivara,  2013).  The  directive  was  extended  to  all  of  DHHS  in  2011. 

We  did  not  include  Army  STARRS  in  this  analysis  because  this  large  study  encompasses  so  many  different  categories 
of  research — as  many  as  11  of  the  12  NAASP  categories — that  it  would  be  difficult  to  interpret  the  results  in  each  cat¬ 
egory.  (The  11  NAASP  categories  are  enhanced  continuity  of  care;  provider  and  gatekeeper  training;  affordable,  accessible, 
and  effective  care;  psychosocial  interventions  for  those  at  risk;  risk  and  protective  factor  interactions;  stigma  reduction; 
population-based  risk  reduction/resilience-building;  prediction  of  imminent  risk;  improved  biological  interventions;  reduc¬ 
tion  in  access  to  lethal  means;  and  population-based  screening). 


Table  2.5 

Number  of  Studies  in  Each  NAASP  Category  and  Funding  Totals 


Overall 

DoD 

VA 

DHHS 

AFSP 

NAASP  Category 

N 

Amount  ($) 

N 

Amount  ($) 

N 

Amount  ($) 

N 

Amount  (S) 

N 

Amount  (S) 

Psychosocial  interventions 

78 

74,640,366 

29 

58,007,419 

6 

3,237,510 

13 

10,943,483 

30 

2,451,954 

Prediction  of  imminent  risk 

48 

31,035,370 

6 

10,124,020 

5 

2,928,482 

13 

16,342,038 

24 

1,640,830 

Risk  and  protective  factors 

58 

75,417,791 

24 

52,752,931 

7 

4,130,210 

27 

16,544,878 

0 

1,989,772 

Biological  interventions 

35 

20,252,292 

3 

13,241,279 

2 

1,255,493 

4 

3,604,115 

26 

2,151,405 

Affordable,  accessible,  and  effective  care 

45 

56,902,577 

30 

53,547,965 

0 

0 

8 

2,653,014 

7 

701,598 

Enhanced  continuity  of  care 

27 

41,399,005 

7 

26,534,042 

4 

2,553,215 

4 

11,142,280 

12 

1,169,468 

Population-based  risk  reduction 

23 

32,228,568 

10 

26,322,413 

2 

972,757 

6 

4,555,098 

5 

378,300 

Provider/gatekeeper  training 

23 

26,274,020 

9 

20,154,686 

0 

0 

7 

5,417,617 

7 

701,717 

Population-based  screening 

11 

7,251,846 

3 

1,656,608 

3 

1,064,207 

5 

4,531,031 

0 

0 

Prevention  of  reattempts 

11 

15,849,481 

2 

10,550,733 

0 

0 

6 

5,087,180 

3 

211,568 

Stigma  reduction 

11 

23,002,803 

7 

19,064,891 

0 

0 

2 

3,777,912 

2 

160,000 

Access  to  lethal  means 

3 

831,609 

0 

0 

1 

387,711 

2 

443,898 

0 

0 

NOTE:  The  table  does  not  include  Army  STARRS,  the  Millennium  Cohort  Study,  or  STRONG  STAR.  Totals  by  category  exceed  total  funding  because  some  studies  are 
counted  in  more  than  one  category. 


16  Developing  a  Research  Strategy  for  Suicide  Prevention  in  the  Department  of  Defense 


Current  Suicide  Prevention  Research  in  the  United  States  That  Is  Directly  Relevant  to  Military  Personnel  17 


Figure  2.4 

Proportion  of  NAASP  Category  Studies  Being  Addressed  by  Different  Funders,  by  Number  of  Studies 


Psychosocial  interventions  (78) 

Prediction  of  imminent  risk  (48) 

Affordable,  accessible,  and 
effective  care  (45) 

Enhanced  continuity  of  care  (27) 

Population-based  risk  reduction/ 
resilience-building  (23) 

Risk  and  protective  factors  (58) 

Provider/gatekeeper  training  (23) 

Stigma  reduction  (11) 

Biological  interventions  (35) 

Prevention  of  reattempts  (11) 

Population-based  screening  (11) 

Reduction  in  access  to 
lethal  means  (3) 

0  20  40  60  80  100 

Percentage 

NOTE:  The  figure  does  not  include  Army  STARRS.  Values  in  parentheses  next  to  each  goal  represent  the  total 
number  of  studies. 
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Amount  of  Funding 

Overall,  across  all  categories  of  funders,  the  most  funding  is  being  spent  on  risk  and  pro¬ 
tective  factor  interactions  ($75.4  million)  and  psychosocial  interventions  for  those  at  risk 
($74.7  million;  see  Table  2.5).  The  next  highest  amount  of  funding  is  devoted  to  studies  of 
affordable,  accessible,  and  effective  care  ($56.9  million)  and  enhanced  continuity  of  care 
($41.4  million).  A  moderate  amount  of  funding  is  devoted  to  studies  of  population-based  risk 
reduction  and  resilience-building,  prediction  of  imminent  risk,  provider  and  gatekeeper  train¬ 
ing,  stigma  reduction,  and  improved  biological  interventions  (listed  here  from  most  to  least 
funding).  The  least  amount  of  funding  is  being  spent  on  studies  on  the  prevention  of  reat¬ 
tempts  ($15.9  million),  population-based  screening  ($7.3  million),  and  reducing  access  to  lethal 
means  ($831,609),  a  category  with  significantly  less  funding  than  the  others. 

Similar  to  the  overall  results,  DoD’s  top  areas  of  funding  are  studies  on  psychoso¬ 
cial  interventions  for  those  at  risk  ($58  million);  affordable,  accessible,  and  effective  care 
($53.5  million);  and  risk  and  protective  factor  interactions  ($52.8  million).  Its  lowest  priorities 
for  funding  are  studies  on  the  prevention  of  reattempts  ($10.6  million)  and  population-based 
screening  ($1.7  million).  There  is  no  study  in  DoD’s  portfolio  focused  on  reducing  access  to 
lethal  means.  The  top  categories  for  VA  funding  are  also  risk  and  protective  factor  interactions 
($4.1  million)  and  improved  psychosocial  intervention  ($3.2  million). 

The  funding  priorities  look  a  little  different  for  studies  funded  by  DHHS  and  AFSP  rela¬ 
tive  to  funding  allocations  in  DoD  and  the  VA.  DHHS  prioritizes  funding  research  on  risk  and 
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protective  factor  interactions  ($16.3  million),  but  unlike  DoD  and  the  VA,  DHHS  allocates 
$16.3  million  to  studies  on  the  prediction  of  imminent  risk,  making  this  the  second  highest 
funding  category.  Furthermore,  studies  of  affordable,  accessible,  and  effective  care  are  a  low 
priority  at  DHHS  {-$2.7  million),  while  they  are  a  relatively  high  priority  for  DoD.  While  the 
AFSP  also  prioritizes  psychosocial  interventions  for  those  at  risk  ($2.5  million),  it  also  prioritizes 
studies  on  improved  biological  interventions  ($2.2  million),  a  topic  that  is  not  a  high  priority 
for  other  funders.  Like  DoD,  the  AFSP  does  not  prioritize  studies  of  population-based  screen¬ 
ing  or  reduced  access  to  lethal  means,  devoting  no  funding  at  all  to  these  areas  of  research. 

As  we  did  earlier,  another  way  to  examine  the  funding  data  is  to  consider  the  proportion 
of  funding  within  each  NAASP  category  that  is  being  provided  by  each  of  the  major  funders 
(Figure  2.5).  The  most  striking  advantage  of  this  perspective  on  the  results  is  that  it  enables  us 
to  easily  see  what  an  important  funder  DoD  is,  accounting  for  more  than  half  of  all  funding 
in  almost  all  of  the  categories.  Conversely,  we  can  see  that  even  though  the  AFSP  funds  the 
largest  number  of  studies  on  biological  interventions,  it  contributes  only  1 1  percent  of  all  fund¬ 
ing  in  this  area  (65  percent  comes  from  DoD).  DoD  is  funding  more  than  three-quarters  of 
all  current  research  on  stigma  reduction;  provider  and  gatekeeper  training;  population-based 
risk  reduction/resilience-building;  affordable,  accessible,  and  effective  care;  and  psychosocial 
interventions.  In  contrast,  other  federal  government  agencies,  such  as  NIH,  are  funding  the 
majority  of  research  on  predicting  imminent  risk  and  population-based  screening,  to  which 
DoD  contributes  less  than  25  percent. 

Figure  2.5 

Proportion  of  NAASP  Category  Funding  Being  Addressed  by  Different  Funders,  by  Amount  Funded 
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parentheses  next  to  each  goal  represent  the  total  amount  funded,  in  $  millions. 
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We  also  examined  the  mean  (i.e.,  average)  funding  amount  for  studies  on  each  topic  (see 
Table  2.6).  The  mean  amount  of  funding  per  study  likely  reflects  both  the  priorities  of  the 
funders  and  the  costs  of  conducting  certain  kinds  of  studies.  Across  all  study  categories,  DoD 
spends,  on  average,  the  most  per  study  (approximately  $1.8  million),  whereas  the  AFSP  funds 
studies  at  an  average  of  $78,000.  It  is  noteworthy  that  the  amount  spent  on  studies  to  prevent 
reattempts  ($5.3  million)  and  improve  biological  therapies  in  DoD  ($4.4  million)  is  much 
greater  than  the  mean  ($1.8  million).  The  mean  funding  level  for  VA  studies  is  $540,000;  for 
DHHS  studies,  it  is  closer  to  $900,000,  with  much  more  per  study  being  spent  on  research 
that  aims  to  enhance  continuity  of  care  ($2.8  million)  and  much  less  being  spent  on  studies 
on  ensuring  affordable,  accessible,  and  effective  care  ($331,627)  and  reducing  access  to  lethal 
means  ($221,949). 

Funding,  by  MOMRP  Category 
Number  of  Studies 

Overall,  across  all  categories  of  funders,  the  most  common  MOMRP  category  of  recent 
research  by  far  was  epidemiology  and/or  basic  science/neurological  mechanisms  (144  studies), 
followed  at  quite  a  distance  by  studies  of  treatment  (71)  and  prevention,  education,  and  train¬ 
ing  (70;  see  Table  2.7).  While  quite  a  bit  of  research  is  also  being  done  on  assessment  (47)  and 


Table  2.6 

Mean  Funding  Amount  for  Studies  in  Each  NAASP  Category,  by  Funder 


Mean  Funding  ($) 

NAASP  Category 

Overall 

DoD 

VA 

DHHS 

AFSP 

All  categories 

1,758,598 

541,708 

881,956 

78,422 

Minimum  grant 

65,786 

69,025 

32,150 

20,000 

Maximum  grant 

9,886,992 

1,098,989 

9,872,609 

240,031 

Psychosocial  interventions 

956,928 

2,000,256 

539,585 

841,806 

81,732 

Prediction  of  imminent  risk 

646,570 

1,687,337 

585,696 

1,257,080 

68,368 

Risk  and  protective  factors 

1,300,307 

2,198,039 

590,030 

612,773 

N/A 

Biological  interventions 

578,637 

4,413,760 

627,747 

901,029 

82,746 

Affordable,  accessible,  and 
effective  care 

1,264,502 

1,784,932 

N/A 

331,627 

100,228 

Enhanced  continuity  of  care 

1,533,296 

3,790,577 

638,304 

2,785,570 

97,456 

Population-based  risk 
reduction/resilience-building 

1,401,242 

2,632,241 

486,379 

759,183 

75,660 

Provider/gatekeeper  training 

1,142,349 

2,239,410 

N/A 

773,945 

100,245 

Population-based  screening 

659,259 

552,203 

354,736 

906,206 

N/A 

Prevention  of  reattempts 

1,440,862 

5,275,367 

N/A 

847,863 

70,523 

Stigma  reduction 

2,091,164 

2,723,556 

N/A 

1,888,956 

80,000 

Access  to  lethal  means 

277,203 

N/A 

387,711 

221,949 

N/A 

NOTE:  The  table  not  include  Army  STARRS,  the  Millennium  Cohort  Study,  or  STRONG  STAR. 


Table  2.7 

Number  of  Studies  in  Each  MOMRP  Category  and  Funding  Totals 


Overall 

DoD 

VA 

DHHS 

AFSP 

MOMRP  Category 

N 

Amount  ($) 

N 

Amount  ($) 

N 

Amount  (S) 

N 

Amount  ($) 

N 

Amount  ($) 

Prevention  training/education 

70 

48,349,971 

27 

37,430,134 

6 

3,049,476 

11 

5,991,756 

26 

1,878,605 

Early  screening/intervention 

44 

40,576,088 

8 

11,764,318 

0 

0 

22 

27,831,254 

14 

980,516 

Assessment 

47 

42,157,964 

10 

20,286,221 

8 

4,664,720 

10 

15,773,247 

19 

1,433,776 

Treatment 

71 

95,944,007 

33 

73,638,120 

3 

1,555,725 

16 

18,963,599 

19 

1,786,563 

Recovery  and  return  to  duty 

1 

85,000 

0 

0 

0 

0 

0 

0 

1 

85,000 

Postvention 

8 

10,259,375 

4 

9,930,842 

0 

0 

0 

0 

4 

328,533 

Epidemiology  and/or  basic  science/ 
neurological  mechanisms 

144 

104,595,649 

38 

73,252,842 

10 

5,921,616 

34 

20,366,322 

62 

5,054,869 

NOTE:  The  table  does  not  include  Army  STARRS.  Totals  by  category  exceed  total  funding  because  some  studies  are  counted  in  more  than  one  category. 
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early  screening/intervention  (44),  there  are  relatively  few  studies  examining  either  postven- 
tion  (8)  or  recovery  and  return  to  duty  (1).  This  pattern  generally  holds  when  we  examine  the 
results  hy  funder,  with  epidemiology  and/or  basic  science/neurological  mechanisms  the  largest 
category  being  investigated  across  all  fundersT 

Figure  2.6  presents  the  proportion  of  research  in  each  MOMRP  category  that  is  being 
funded  by  each  of  the  major  funders.  This  hgure  shows  that  DoD  is  responsible  for  half  of  all 
studies  on  postvention  (with  the  AFSP  responsible  for  the  remaining  half)  and  almost  half  of 
all  treatment  studies. 

Amount  of  Funding 

As  shown  in  Table  2.7,  comparable  amounts  are  being  spent  on  epidemiology  and/or  basic  sci¬ 
ence/neurological  mechanisms  ($105  million)  and  treatment  ($96  million).  In  contrast,  only 
$85,000  is  being  spent  on  studies  related  to  recovery  and  return  to  duty.  There  is  roughly 
$10  million  being  spent  on  studies  related  to  postvention  and  just  under  $50  million  being 
spent  individually  on  prevention  education  and  training,  assessment,  and  early  screening/ 
intervention. 

Figure  2.7  reiterates  that  DoD  is  a  large  funder  of  suicide  prevention  research,  accounting 
for  more  than  half  of  the  research  funds  in  four  of  the  seven  categories.  Then  again,  the  single 
study  on  recovery  and  return  to  duty  is  being  funded  by  the  AFSP,  whereas  DHHS  is  respon¬ 
sible  for  69  percent  of  funds  allocated  to  early  screening  and  intervention. 

It  is  also  interesting  to  consider  the  mean  (i.e.,  average)  funding  amount  allocated  by  dif¬ 
ferent  entities  (see  Table  2.8).  We  already  discussed  total  funding  amounts  by  agency.  Within 

Figure  2.6 

Proportion  of  MOMRP  Category  Studies  Being  Addressed  by  Different  Funders, 
by  Number  of  Studies 
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^  Once  again,  we  did  not  include  Army  STARRS  in  this  analysis  because  that  large  study  encompasses  so  many  categories 
that  it  muddies  the  interpretation  of  the  overall  results.  This  study  falls  under  all  seven  MOMRP  categories  described  here. 
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Figure  2.7 

Proportion  of  MOMRP  Category  Studies  Being  Addressed  by  Different  Funders, 
by  Amount  Funded 
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NOTE:  The  figure  does  not  include  Army  STARRS,  the  Millennium  Cohort  Study,  or  STRONG  STAR.  Values  in 
parentheses  next  to  each  goal  represent  the  total  amount  funded,  in  $  millions. 
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Table  2.8 

Mean  Funding  Amount  for  Studies  in  Each  MOMRP  Category,  by  Funder 


Mean  Funding  ($) 

MOMRP  Category 

Overall 

DoD 

VA 

DHHS 

AFSP 

Prevention  training/education 

690,714 

1,386,301 

508,246 

544,705 

72,254 

Early  screening/intervention 

922,184 

1,470,540 

N/A 

1,265,057 

70,037 

Assessment 

896,978 

2,028,622 

583,090 

1,577,325 

75,462 

Treatment 

1,351,324 

2,231,458 

518,575 

1,185,225 

94,030 

Recovery  and  return  to  duty 

85,000 

N/A 

N/A 

N/A 

85,000 

Postvention 

1,282,422 

2,482,711 

N/A 

N/A 

82,133 

Epidemiology  and/or  basic  science/ 
neurological  mechanisms 

726,359 

1,927,706 

592,162 

599,009 

81,530 

NOTE:  The  table  does  not  include  Army  STARRS,  the  Millennium  Cohort  Study,  or  STRONG  STAR. 


the  MOMRP  categories,  it  is  noteworthy  that  studies  on  treatment  and  postvention  have, 
on  average,  the  highest  levels  of  funding.  Although  there  are  only  eight  postvention  studies, 
their  high  value  is  driven  hy  DoD  funding:  The  four  studies  that  DoD  has  funded  related 
to  postvention  average  roughly  $2.5  million.  The  mean  funding  level  for  studies  funded  hy 
DoD  ranges  from  $1.4  million  to  $2.5  million,  while  DHHS  studies  ranged  from  approxi¬ 
mately  $500,000  to  $1.6  million,  depending  on  the  category.  Mean  funding  levels  in  DHHS 
were  highest  for  assessment  and  early  screening  and  intervention,  whereas  DoD  funding  levels 
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were  highest  for  postvention,  treatment,  and  assessment.  Across  MOMRP  categories,  VA  stud¬ 
ies  average  between  $500,000  and  $600,000,  while  AFSP  studies  across  categories  average 
between  $70,000  and  $100,000. 


Conclusion 

We  catalogued  current  suicide  prevention  research  that  is  relevant  to  military  personnel,  includ¬ 
ing  studies  of  comparable  civilian  samples.  We  found  that  DoD  is  the  single  largest  funder  of 
suicide  prevention  research,  having  recently  funded  61  studies  at  a  cost  of  more  than  $100  mil¬ 
lion.  This  figure  does  not  include  Army  STARRS,  a  $65  million  study  funded  jointly  by  DoD 
and  NIMH.  While  DoD  is  the  largest  funder  of  suicide  prevention  research  with  relevance  to 
military  personnel  in  the  United  States,  this  total  represents  a  little  less  than  one-third  of  all 
relevant  studies. 

Although  we  detailed  results  for  all  the  major  funders,  in  the  discussion,  we  focus  pri¬ 
marily  on  DoD  and  overall  trends.  With  respect  to  the  NAASP  categories,  current  suicide 
prevention  research  funded  by  DoD  and  other  funders  tends  to  focus  on  identifying  who 
dies  by  suicide  (i.e.,  risk  and  protective  factor  interactions),  psychotherapeutic  interventions  to 
treat  individuals  at  risk  for  suicide  (i.e.,  psychosocial  interventions),  and  ensuring  that  those  at 
risk  can  access  affordable,  accessible,  and  effective  care.  This  is  true  whether  we  focus  on  the 
number  of  studies  funded  or  the  total  amount  of  funding  devoted  to  these  areas  of  research. 

In  contrast,  relatively  few  studies  funded  by  DoD  and  other  entities  focus  on  preventing 
reattempts  among  those  who  have  previously  attempted  suicide  or  the  reduction  in  access  to 
lethal  means,  and  these  were  also  areas  with  relatively  low  amounts  of  funding.  Interestingly, 
although  it  was  an  area  of  low  overall  spending,  DoD’s  highest  level  of  mean  funding  per  study 
was  for  studies  of  the  prevention  of  reattempts. 

When  MOMRP  categories  are  used,  we  see  that  the  most  studies  by  far  are  being  con¬ 
ducted  on  who  is  at  risk  of  dying  by  suicide  (i.e.,  epidemiology  and/or  basic  science/neurological 
mechanisms).  This  is  the  case  both  overall  and  for  studies  funded  by  DoD  specifically.  There 
are  also  a  large  number  of  studies  focusing  on  treatment  and  prevention  training  and  educa¬ 
tion.  Similarly,  the  most  funding  by  far  is  being  spent  on  epidemiology  and/or  basic  science/ 
neurological  mechanisms  and  treatment,  with  a  moderate  amount  of  spending  on  prevention 
education  and  training,  as  well  as  assessment.  Among  DoD  studies,  mean  funding  per  study 
was  highest  for  studies  of  treatment,  epidemiology  and  basic  science,  and,  surprisingly  postven¬ 
tion,  despite  the  low  level  of  overall  funding  going  to  the  latter  research  area.  Indeed,  relatively 
few  studies  are  examining  either  postvention  or  recovery  and  return  to  duty,  and  there  is  very 
little  spending  in  these  areas  as  well.  While  these  areas  were  not  high  research  funding  priori¬ 
ties  for  DoD,  it  is  notable  that  federal  government  agencies,  such  as  NIH,  did  not  fund  a  single 
study  in  either  of  these  areas,  suggesting  that  if  DoD  does  value  research  in  these  areas,  it  will 
likely  have  to  fund  it.  Indeed,  recovery  and  return  to  duty  may  be  particularly  central  to  DoD’s 
mission  (despite  the  relatively  low  levels  of  funding  for  research  in  this  area),  but  this  topic  may 
not  be  not  viewed  as  a  key  area  of  research  for  other  funders. 

It  is  important  to  note  the  limitations  of  our  abstraction  process  and  analysis.  First,  the 
analysis  does  not  provide  a  comprehensive  summary  of  all  suicide  prevention  data  in  that  it 
focuses  on  recent  studies  that  are  relevant  to  military  personnel  and  excludes  studies  prior  to 
2005,  those  of  young  children,  and  those  of  individuals  with  serious  mental  illness.  In  addi- 
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tion,  many  relevant  studies  were  broad  in  their  focus,  examining  issues  beyond  suicide  pre¬ 
vention,  and  it  was  not  possible  to  determine  the  proportion  of  funds  that  went  specifically  to 
suicide  prevention  research.  Thus,  all  the  funding  analyses  necessarily  lack  some  precision  and 
may  overestimate  spending  on  suicide  prevention  research.  Finally,  many  studies  fell  under 
more  than  one  category  of  NAASP  or  MOMRP  research.  It  is  not  possible  to  determine  the 
amount  of  money  spent  on  research  goals  in  each  specific  category,  so  the  funds  are  often 
counted  in  multiple  categories.  Thus,  funding  estimates  for  research  categories  lack  precision 
and  are  likely  overestimates  of  how  much  funding  is  being  spent  on  a  given  type  of  research. 
Appendix  B  presents  the  results  under  an  alternative  system  of  allocating  funds  uniformly 
across  these  categories.  The  only  noticeable  difference  is  that  overall  spending  for  enhanced 
continuity  of  care  ranks  relatively  higher  when  totals  are  allocated  to  each  category,  and  the 
relative  ranking  of  this  category  is  lower  when  funds  are  dispersed  equally  across  categories, 
signifying  that  funding  in  this  category  is  often  allocated  as  part  of  research  studies  that  cut 
across  multiple  categories.  Otherwise,  the  general  themes  and  conclusions  presented  here  are 
consistent  across  methods. 

These  results  regarding  research  priorities  should  be  interpreted  with  the  caveat  that  they 
do  not  include  the  major  Army  STARRS  study,  which  devotes  $65  million  to  examining  11  of 
12  NAASP  categories  of  suicide  prevention  research  and  all  seven  MOMRP  categories.  When 
we  report  that  funding  is  low  in  an  area,  it  does  not  take  into  account  the  fact  that  Army 
STARRS  is  likely  conducting  research  in  the  area.  However,  it  is  impossible  to  determine  the 
amount  of  Army  STARRS  funding  that  goes  to  each  research  priority.  Furthermore,  even  if 
Army  STARRS  is  examining  a  relevant  topic,  it  may  still  be  helpful  for  other  research  studies 
with  different  viewpoints  to  examine  the  same  topic,  because  knowledge  tends  to  be  best  accu¬ 
mulated  by  combining  the  results  of  numerous  studies  with  differing  approaches — a  topic  to 
which  we  return  in  Chapter  Six. 


CHAPTER  THREE 
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In  this  chapter,  we  present  DoD’s  needs  for  suicide  prevention  research  as  perceived  hy  repre¬ 
sentatives  who,  because  of  their  organizational  affiliation,  play  a  role  in  DoD’s  suicide  preven¬ 
tion  activities.  To  do  so,  we  huilt  on  the  aspirational  research  goals  established  by  the  NAASP 
and  conducted  an  online  consensus-building  exercise.  We  ranked  suicide  prevention  priorities 
across  five  domains:  importance,  cultural  acceptability,  effectiveness,  cost,  and  future  learning 
potential.  Although  a  less-than-ideal  participation  rate  limits  the  conclusions  we  can  make, 
the  process  for  ranking  priorities  enabled  us  to  make  sensible  and  strong  conclusions,  and  the 
exercise  could  be  replicated  by  DoD  to  boost  participation. 


Background 

Underpinning  any  strategic  research  agenda  is  an  overarching  set  of  research  objectives  and 
goals,  an  understanding  of  the  current  state  of  knowledge,  and  insights  from  stakeholders 
about  perceived  priorities  for  additional  investment.  As  part  of  its  own  efforts  to  articulate  a 
national  research  agenda  on  suicide  prevention,  the  NAASP  Research  Prioritization  Task  Force 
employed  a  three-part,  web-based  elicitation  process  to  gather  input  from  stakeholders.  This 
process  was  designed  to  inform  the  selection  of  aspirational  research  goals  using  four  rounds  of 
data  collection  and  decisionmaking  through  a  process  known  as  the  Delphi  method. 

The  task  force  solicited  ideas  about  research  goals  from  more  than  700  survey  partic¬ 
ipants.  After  additional  rounds  of  data  collection  and  iteration  among  survey  participants, 
the  task  force  identified  the  12  most  frequently  mentioned  goals.  In  additional  rounds  of  the 
online  survey,  these  goals  were  then  ranked,  rated,  and  discussed.  The  final  results  of  the  task 
force’s  stakeholder  survey  yielded  a  three-tiered  list  of  aspirational  goals  to  guide  the  creation 
of  the  national  research  agenda  (see  Table  3.1).  This  list  served  as  the  starting  point  for  our  own 
process  of  engaging  stakeholders  and  collecting  feedback  on  research  priorities. 

While  the  task  force’s  process  was  open  to  DoD  stakeholder  participation,  the  data  were 
not  collected  in  a  manner  that  allowed  us  to  isolate  the  input  and  priorities  of  DoD  stakehold¬ 
ers  from  those  of  other,  non-DoD  stakeholders.  Therefore,  we  sought  to  replicate  the  approach 
used  by  the  task  force  to  identify  specific  priorities  among  DoD  stakeholders.  We  employed 
the  same  procedures  used  by  the  task  force,  including  identical  online  elicitation  software  and 
processes.  We  took  these  goals  and  asked  stakeholders  to  consider  their  merits  on  five  grounds: 

1.  Overall  importance.  In  the  absence  of  asking  participants  to  rank  the  goals,  we  asked 
them  to  provide  their  opinions  about  the  overall  importance  of  each  particular  strategy 
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Table  3.1 

Aspirational  Goals  Identified  by  NAASP  Research 
Prioritization  Task  Force 

Tier  Goal 

1  Prevention  of  reattempts 

1  Enhanced  continuity  of  care 

1  Provider  and  gatekeeper  training 

1  Affordable,  accessible,  and  effective  care 

2  Psychosocial  interventions  for  those  at  risk 

2  Risk  and  protective  factor  interactions 

2  Stigma  reduction 

2  Population-based  risk  reduction/resilience-building 

2  Prediction  of  imminent  risk 

3  Improved  biological  interventions 

3  Reduction  in  access  to  lethal  means 

3  Population-based  screening 


in  suicide  prevention.  The  purpose  was  to  help  participants  differentiate  among  the 
goals. 

2.  Ejfectiveness.  Participants  were  asked  to  gauge  how  effective,  according  to  their  knowl¬ 
edge  of  existing  research  or  experience,  the  particular  strategy  is  for  preventing  suicide. 

3.  Cultural  acceptability.  One  of  the  critical  components  of  the  success  of  any  prevention 
strategy  is  whether  it  will  be  considered  culturally  acceptable  within  a  given  popula¬ 
tion  or  setting.  Therefore,  participants  were  asked  to  provide  feedback  on  the  cultural 
acceptability  of  the  particular  strategy  in  a  military  setting. 

4.  Cost.  A  key  factor  in  choosing  among  prevention  strategies  is  the  cost  associated  with  its 
implementation.  Some  approaches  are  likely  to  be  more  costly  than  others.  Thus,  par¬ 
ticipants  were  asked  to  evaluate  the  cost  associated  with  each  strategy. 

5.  Learning  potential.  Participants  were  asked  to  consider  the  learning  potential  associated 
with  each  strategy.  When  it  comes  to  strategies  that  have  been  tested  quite  a  bit,  there 
may  be  a  limit  to  how  much  new  information  can  be  gleaned  from  further  research; 
conversely,  other  strategies  may  have  a  large  learning  potential,  either  because  little  is 
known  or  because  they  are  being  applied  differently  than  in  prior  studies. 


Methods 

RAND  ExpertLens  is  an  evidence-based  process  to  develop  consensus  among  a  diverse  set  of 
participants  (Dalai  et  ah,  2011).  It  uses  the  Delphi  method  to  gather  numeric  answers  (e.g.,  rat- 
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ings)  and  discussion  comments  to  facilitate  consensus.'  By  combining  numeric  answers  with 
discussion  comments,  RAND  ExpertLens  not  only  identifies  areas  of  agreement  in  large  and 
diverse  groups,  but  it  also  allows  researchers  to  understand  where  participants  disagree 
and  why  their  perspectives  vary.  The  anonymity  afforded  by  the  online  process  further  encour¬ 
ages  participants  to  be  honest  in  expressing  their  ideas  and  to  evaluate  other  participants’  com¬ 
ments  based  on  their  substance  rather  than  the  social  status  or  interpersonal  characteristics  of 
participating  individuals. 

Creating  Aspirational  Goals 

The  first  step  in  replicating  this  process  for  DoD  was  to  review  and  adapt  the  12  aspirational 
goals  identified  earlier  in  this  report.  The  RAND  team  edited  the  12  aspirational  goals  in  an 
effort  to  make  them  more  specific  and  applicable  to  the  DoD  community  and  setting.  This 
involved  adding  specific  action- oriented  verbs  to  help  participants  understand  what  would  be 
achieved  if  the  goal  were  implemented.  It  should  be  noted  that,  in  one  case  (risk  and  protective 
factor  interactions),  we  did  not  provide  a  matching  goal.  Additionally,  the  original  goal  “pro¬ 
vider  and  gatekeeper  training”  was  broken  into  two  goals:  (1)  ensure  non-health  professionals 
(i.e.,  noncommissioned  officers,  chaplains)  who  come  into  contact  with  suicidal  individuals  are 
trained  to  identify,  care  for,  and  refer  persons  at  risk,  and  (2)  train  health  care  professionals  to 
identify  those  at  risk  for  suicide  and  to  manage  their  treatment.  The  original  aspirational  goals 
and  the  adaptations  that  were  used  for  the  RAND  ExpertLens  elicitation  process  are  presented 
in  Table  3.2. 

It  is  important  to  recognize  the  distinction  between  research  and  implementation  priori¬ 
ties.  These  12  aspirational  goals  could  be  considered  both;  however,  the  ultimate  goal  of  this 
research  was  to  elicit  feedback  relevant  to  the  prioritization  and  allocation  of  the  research  port¬ 
folio,  which  is  a  subset  of  the  overall  suicide  prevention  portfolio.  While  policy  and  program 
officials  may  wish  to  prioritize  activities  that  have  the  greatest  potential  to  prevent  suicides 
(e.g.,  those  that  have  demonstrated  effectiveness  from  prior  research)  among  implementation 
priorities,  research  priorities  may  be  set  according  to  where  there  is  the  greatest  learning  poten¬ 
tial  or  need  for  studies  to  facilitate  adaptation  to  new  populations  or  settings.  Using  the  criteria 
employed  by  the  NAASP  allowed  us  to  examine  ratings  along  several  dimensions  to  inform 
how  stakeholders  viewed  each  goal.  Research  and  implementation  goals  are  intrinsically  linked 
because  the  ultimate  goal  is  to  create  an  effective  suicide  prevention  strategy. 

Selection  of  Stakeholder  Participants 

The  RAND  ExpertLens  process  is  designed  to  gather  insight  and  feedback  from  among  a 
group  of  diverse  stakeholders  from  different  backgrounds  and  with  different  levels  of  expertise. 
Therefore,  we  sought  to  identify  a  group  of  participants  that  would  cut  across  DoD  organiza¬ 
tions,  service  branches,  and  defense  agencies.  We  worked  with  the  sponsoring  office  to  identify 
potential  participants  from  across  relevant  DoD  organizations.  We  also  sought  representation 
from  each  of  the  services  and  defense  agencies,  including  individuals  responsible  for  suicide 
prevention  programming  and  oversight  and  for  behavioral  and  psychological  health  program- 


*  The  Delphi  method  is  a  structured  process  for  deriving  expert  consensus  and  was  used  in  at  least  1,386  published 
research  studies  between  2000  and  2004  (Landeta,  2006).  It  is  the  leading  method  used  to  develop  health  care  quality 
indicators  (Boulkedid  et  ah,  2011)  and  research  methodologies  (Verhagen  et  ah,  1998)  and  has  recently  been  applied  in  the 
social  sciences  for  technical  forecasting  (Landeta,  2006) 
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Table  3.2 

DoD  Aspirational  Goals  for  RAND  ExpertLens 


NAASP  Aspirational  Goal 

DoD  Aspirational  Goal 

RAND  Shorthand 

Population-based  risk  reduction/ 
resilience-building 

Implement  population-based  programs 
that  reduce  suicide  risk  factors  and  build 
resilience. 

Risk  reduction 

Provider  and  gatekeeper  training 

Ensure  non-health  professionals 
(i.e.,  noncommissioned  officers,  chaplains) 
who  come  in  contact  with  suicidal  individuals 
are  trained  to  identify,  care  for,  and  refer 
persons  at  risk. 

Gatekeeper  training 

Provider  and  gatekeeper  training 

Train  health  care  professionals  to  identify 
those  at  risk  for  suicide  and  to  manage  their 
treatment. 

Provider  training 

Stigma  reduction 

Encourage  service  members  and  their 
families  to  be  knowledgeable  about  and 
proactively  seek  treatment. 

Help-seeking 

Affordable,  accessible,  and  effective 
care 

Deliver  high-quality  treatments  for  mental 
illnesses  (e.g.,  depression,  PTSD)  that  are 
associated  with  suicide. 

Quality  care 

Population-based  screening 

Conduct  population-based  screening  to 
identify  those  at  risk  for  suicide. 

Screening 

Reduction  in  access  to  lethal  means 

Reduce  service  members'  access  to  the 
means  that  they  might  use  to  take  their 
own  lives. 

Reduced  access 

Psychosocial  interventions  for  those 
at  risk 

Improve  psychosocial  interventions  used  by 
clinicians  (e.g.,  psychiatrists,  psychologists, 
social  workers)  to  identify  and  treat  those  at 
risk  for  suicide. 

Psychosocial  interventions 

Improved  biological  interventions 

Identify  biological  interventions  clinicians 
could  use  to  treat  suicidal  behavior. 

Biological  interventions 

Prediction  of  imminent  risk 

Develop  strategies  to  predict  which 
individuals  are  at  imminent  risk  of  suicide. 

Prediction 

Enhanced  continuity  of  care 

Achieve  continuity  of  care  between 
providers,  across  installations,  and  with  the 
civilian  and  VA  systems. 

Continuity  of  care 

Prevention  of  reattempts 

Implement  strategies  to  prevent  suicide 
reattempts. 

Prevent  reattempts 

Risk  and  protective  factor 
interactions 

[Excluded  because  we  could  not  equate  it 
with  an  aspirational  goal.] 

NOTE:  Goals  are  worded  verbatim  from  the  RAND  ExpertLens  exercise.  The  goal  "risk  and  protective  factor 
interactions"  was  excluded  because  we  could  not  equate  it  with  a  specific  aspirational  goal.  RAND  shorthand 
terms  were  not  included  in  the  RAND  ExpertLens  process  but  are  included  here  for  reference  because  we  use 
these  shorthand  descriptions  for  the  remainder  of  this  chapter. 
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ming,  service  delivery,  and  family  support  programming  and  oversight.  Potential  subject- 
matter  experts  also  included  DoD  intramural  researchers  working  in  the  area  of  suicide  preven¬ 
tion.  It  should  be  noted  that  we  hrst  identified  the  groups,  offices,  organizations,  and  specific 
positions  that  we  wanted  to  include  in  our  sample  of  experts;  then,  the  sponsoring  office  identi¬ 
fied  the  specific  individuals  who  met  those  criteria  as  of  April  2013.  This  process  yielded  a  list 
of  104  DoD  stakeholders  representing  the  following  offices:^ 

•  Office  of  the  Assistant  Secretary  of  Defense  for  Health  Affairs 

•  Office  of  the  Assistant  Secretary  of  Defense  for  Legislative  Affairs 

•  Office  of  the  Assistant  Secretary  of  Defense  for  Reserve  Affairs 

•  Office  of  the  Assistant  Secretary  of  Defense  for  Readiness  and  Force  Management 

•  Office  of  the  Deputy  Assistant  Secretary  of  Defense  for  Military  Family  and  Community 
Policy 

•  DoD  Office  of  the  Inspector  General 

•  U.S.  Joint  Chiefs  of  Staff 

•  U.S.  Army  (invitations  were  sent  to  representatives  in  offices  related  to  manpower,  the 
Army  Surgeon  General,  family  support,  the  Ready  and  Resilient  Campaign,  suicide  pre¬ 
vention,  Army  STARRS,  and  Army  Medical  Research  and  Materiel  Command) 

•  U.S.  Navy  (invitations  were  sent  to  representatives  in  offices  related  to  manpower,  the 
Navy  Surgeon  General,  family  support,  suicide  prevention,  and  naval  health  research) 

•  U.S.  Air  Force  (invitations  were  sent  to  representatives  in  offices  related  to  manpower,  the 
Air  Force  Surgeon  General,  and  family  support) 

•  U.S.  Marine  Corps  (invitations  were  sent  to  representatives  in  offices  related  to  manpower 
and  personnel  and  family  support) 

•  Defense  Suicide  Prevention  Office 

•  Defense  Centers  of  Excellence  for  Psychological  Health  and  Traumatic  Brain  Injury 

•  Defense  Advanced  Research  Projects  Agency 

•  Armed  Forces  Chaplains  Board 

Implementation  of  the  DoD  Stakeholder  Elicitation  Process 

In  preparation  for  the  RAND  ExpertLens  process,  we  sent  an  introduction  email  to  the 
104  potential  participants  from  across  the  identified  DoD  organizations.  The  email  informed 
them  of  the  objective  of  the  RAND  study,  provided  an  overview  of  the  process,  and  invited 
their  participation  in  the  RAND  ExpertEens  elicitation.  The  email  also  included  a  memo 
signed  by  Frederick  Vollrath,  Principal  Deputy  Assistant  Secretary  of  Defense  for  Readiness 
and  Force  Management.  ^  This  memo  explained  the  purpose  of  the  online  elicitation  process 
and  encouraged  their  participation.  Participants  then  received  official  invitations  from  the 
RAND  ExpertEens  administrator  to  log  into  the  RAND  ExpertEens  system.  These  emails 
provided  specific  instructions,  and  each  potential  participant  was  assigned  a  unique  username 
and  password.  Reminder  emails  were  sent  to  participants  throughout  the  process  to  encour¬ 
age  participation  and  to  remind  them  of  the  data  collection  windows.  At  the  initiation  of  each 
wave  of  data  collection,  they  received  emails  that  included  the  URE  for  the  ExpertEens  portal 


^  There  were  24  participants. 

3  Vollrath  was  appointed  Assistant  Secretary  of  Defense  for  Readiness  and  Force  Management  in  April  2013. 
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and  login  instructions  (including  username).  The  opportunity  to  participate  in  each  wave  was 
independent  of  participation  in  prior  waves. 

Implementing  the  RAND  ExpertLens  Process 

The  online  expert  panel  process  was  intended  to  gather  input  from  subject-matter  experts  and 
program  officials  from  across  DoD  to  identify  research  priorities  that  could  inform  the  devel¬ 
opment  of  a  unified,  strategic  research  agenda  on  military  suicide  prevention.  The  expert  panel 
process  was  administered  online  and  involved  three  rounds  of  data  collection  over  a  period  of 
approximately  five  weeks. 

In  round  1,  participants  were  asked  to  answer  a  set  of  questions  about  research  objec¬ 
tives  and  priorities  for  suicide  prevention  within  DoD  (see  Table  3.2).  Using  a  ten-point  Likert 
scale,  participants  were  asked  to  rate,  based  on  their  own  perceptions  and  understanding,  each 
goal  according  to  five  criteria:  perceived  importance,  perceived  effectiveness,  cultural  accep¬ 
tance,  cost,  and  learning  potential.  These  questions  and  the  corresponding  scales  are  shown  in 
Table  3.3. 

At  the  end  of  round  1,  we  also  gathered  some  basic  demographic  information  about  the 
participants,  including  branch  of  service,  age  group,  gender,  race,  education  level,  profession 
(e.g.,  health  care  provider,  researcher),  and  whether  they  had  a  friend  or  relative  who  had  ever 
attempted  or  died  by  suicide. 

In  round  2,  we  presented  participants  with  a  summary  of  the  group’s  responses  from 
round  1.  We  then  invited  participants  to  comment  on  their  own  and  their  colleagues’  responses 


Table  3.3 

Evaluation  Criteria  That  ExpertLens  Participants  Were  Asked  to  Employ  for  Aspirational  Goals 


Criterion  1:  How  important  overall  is  pursuing  this  strategy  in  reducing  the  military  suicide  rate? 

Not 

important 

1  2  3  4  5  6 

7 

8 

9 

Very 

important 

10 

Criterion  2:  How  effective  is  this  goal  in  reducing  suicide  attempts? 

Not 

effective 

1  2  3  4  5  6 

7 

8 

9 

Very 

effective 

10 

Criterion  3:  How  culturally  acceptable  in  DoD  are  approaches  that  implement  this  strategy? 

Not 

culturally 

appropriate 

1  2  3  4  5  6 

7 

8 

9 

Very 

culturally 

appropriate 

10 

Criterion  4:  How  expensive  is  it  to  implement  this  strategy? 

Not 

expensive 

1  2  3  4  5  6 

7 

8 

9 

Very 

expensive 

10 

Criterion  5:  How  much  could  we  learn  in  the  next  five  years  by  funding  research  pursuant  to  this  goal? 

Learn  very 
little 

1  2  3  4  5  6 

7 

8 

9 

Learn 
a  lot 

10 
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in  an  online  discussion  forum.  This  round  afforded  participants  the  opportunity  to  post  their 
own  discussion  questions  and  to  engage  in  an  anonymous  online  discussion  with  other  partici¬ 
pants.  Two  RAND  moderators  prompted  discussions  and  responded  to  comments  to  encour¬ 
age  further  dialogue  and  help  identify  reasons  for  the  variation  observed  in  round  1. 

Finally,  in  round  3,  we  provided  participants  with  the  opportunity  to  change  their  round 
1  answers.  At  the  end  of  round  3,  we  asked  participants  to  complete  a  brief  questionnaire  about 
their  experience  with  the  RAND  ExpertLens  process. 

Analysis 

Upon  completion  of  all  three  rounds  of  ExpertLens,  the  study  team  had  two  sources  of  rat¬ 
ings  (from  rounds  1  and  3)  for  each  of  the  12  aspirational  research  goal  across  each  of  the  five 
domains.  We  used  statistical  analysis  of  these  ratings  to  extract  an  overall  intrinsic  score  for 
each  strategy  based  on  group  opinion.  This  intrinsic  score  accounts  for  the  “noisy”  responses 
from  each  rater  by  weighing  raters  who  exhibited  higher  precision  in  their  responses  more 
heavily  than  lower-precision  raters,  whom  we  assume  had  less  expertise.  More  details  about  the 
approach  can  be  found  in  the  Appendix  B. 


Results 

Participant  Response 

Of  the  104  representatives  invited  to  participate,  24  (23  percent)  participated  in  round  1  of 
the  elicitation  process.  Round  2  offered  participants  the  opportunity  to  review  the  results 
from  round  1  and  interact  with  other  participants  in  a  dialogue  about  divergent  perspectives. 
RAND  staff  prepopulated  the  discussion  with  two  questions.  By  the  end  of  the  week,  there 
were  seven  discussion  threads  on  topics  ranging  from  perceptions  of  the  most  important  goal 
to  concerns  about  the  costs  associated  with  the  strategies  in  the  current  fiscal  climate  and  que¬ 
ries  asking  why  participants  thought  variation  in  responses  existed  where  it  did.  There  was  at 
least  one  reply  posted  to  each  question.  In  total,  16  of  the  24  round  1  participants  logged  into 
round  2,  but  only  five  posted  comments  or  questions.  Seventeen  of  those  who  participated  in 
round  1  (70.8  percent)  participated  in  round  3.  Descriptive  characteristics  of  these  participants 
are  provided  in  Table  3.4. 

Although  the  overall  number  of  participants  was  low,  the  panel  consisted  primarily  of 
uniformed  service  members  (versus  DoD-employed  civilians)  affiliated  with  three  service 
branches.  (We  had  no  representation  from  the  Marine  Corps  or  Coast  Guard.)  However,  with 
respect  to  organizational  affiliation,  we  had  representatives  from  the  Office  of  the  Secretary  of 
Defense;  the  Army,  Navy,  Air  Force,  and  Marine  Corps;  and  the  Joint  Chiefs  of  Staff.  More 
than  half  of  those  who  responded  reported  that  they  had  a  friend  or  relative  who  had  attempted 
or  died  by  suicide.  Their  reactions  to  participating  in  the  consensus-building  exercise  are  pre¬ 
sented  in  Appendix  C. 

Ranking  Across  the  Five  Domains 

We  used  the  intrinsic  scores  generated  through  the  elicitation  process  to  rank  each  aspirational 
goal  across  each  of  the  five  domains.  These  rankings  are  provided  in  Table  3.5. 

We  observed  variation  across  the  five  evaluation  criteria  in  terms  of  participants’  responses. 
That  is,  no  one  intervention  strategy  achieved  the  same  rank  across  all  criteria.  However, 
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Table  3.4 

Participant  Demographics 


Characteristic 

N 

(total  =  24) 

Uniformed  service  members 

19 

Army 

10 

Navy 

4 

Air  Force 

4 

Marine  Corps 

0 

Coast  Guard 

0 

Public  health  service 

1 

Organizational  affiliation 

Office  of  the  Secretary  of  Defense 

4 

Army 

8 

Navy 

4 

Air  Force 

1 

Marine  Corps 

1 

Defense  agency  (Defense  Advanced  Research  Projects 
Agency,  Defense  Threat  Reduction  Agency) 

0 

Joint  Chiefs  of  Staff 

2 

Race/ethnicity 

White 

21 

Nonwhite 

3 

Hispanic  origin 

1 

Gender  (male) 

14 

Education  level  (more  than  4  years  of  college) 

21 

Professional  affiliation 

Health  care  provider 

12 

Researcher 

3 

Age 

18-24 

0 

24-44 

4 

45-64 

18 

65+ 

1 

Had  a  friend  or  relative  who  attempted  or  died  by 
suicide 

13 

NOTE:  Totals  do  not  sum  due  to  missing  data. 
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Table  3.5 

RAND  ExpertLens  Ranking  of  Aspirational  Goals  by  Intrinsic  Score  After  Round  3 


Rank 

Importance 

Effectiveness 

Cultural 

Acceptance 

Cost 

(lowest  to  highest) 

Learning 

Potential 

1 

Provider  training 

Provider  training 

Gatekeeper 

training 

Gatekeeper 

training 

Quality  care 

2 

Continuity  of  care 

Help-seeking 

Continuity  of  care 

Reduced  access 

Continuity  of  care 

3 

Gatekeeper 

training 

Quality  care 

Provider  training 

Help-seeking 

Prediction 

4 

Help-seeking 

Gatekeeper 

training 

Prevent  reattempts 

Prevent  reattempts 

Gatekeeper 

training 

5 

Quality  care 

Continuity  of  care 

Risk  reduction 

Risk  reduction 

Psychosocial 

interventions 

6 

Prediction 

Prevent  reattempts 

Quality  care 

Prediction 

Provider  training 

7 

Prevent  reattempts 

Psychosocial 

interventions 

Help-seeking 

Provider  training 

Help-seeking 

8 

Risk  reduction 

Prediction 

Psychosocial 

interventions 

Continuity  of  care 

Risk  reduction 

9 

Psychosocial 

interventions 

Reducing  access 

Biological 

interventions 

Psychosocial 

interventions 

Prevent  reattempts 

10 

Reduced  access 

Risk  reduction 

Prediction 

Biological 

interventions 

Screening 

11 

Screening 

Biological 

interventions 

Screening 

Quality  care 

Biological 

interventions 

12 

Biological 

interventions 

Screening 

Reduced  access 

Screening 

Reduced  access 

some  patterns  did  emerge.  Gatekeeper  training  was  ranked  as  one  of  the  top  four  strategies 
across  all  domains  (including  the  least  expensive),  whereas  screeningz.n^  biological  interventions 
were  both  rated  as  being  costly  and  at  the  bottom  of  the  rankings  in  terms  of  importance, 
effectiveness,  and  learning  potential.  We  provide  an  overview  of  other  key  findings  across  each 
category  in  the  following  sections. 

Perceived  Importance 

DoD  participants  ranked  provider  training  highest  in  terms  of  perceived  importance,  followed 
by  ensuring  continuity  of  care.  Among  our  DoD  participants,  biological  interventions  were  rated 
lowest  as  a  prevention  strategy  for  suicide. 

Effectiveness 

Similarly,  in  terms  of  effectiveness  provider  trainingv^-AS,  rated  highest  by  DoD  participants  fol¬ 
lowed  by  help-seeking  and  quality  of  care.  Thus,  the  top  three  strategies  in  terms  of  perceived 
effectiveness  as  rated  by  DoD  participants  involve  improving  the  provision  of  or  access  to  high- 
quality  health  care  for  those  at  risk.  As  with  perceived  importance,  biological  interventions  and 
screening  ranked  lowest  in  terms  of  effectiveness. 
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Cultural  Acceptability 

Participants  ranked  gatekeeper  training  highest  in  terms  of  cultural  acceptahility  in  military 
settings.  This  is  a  widely  used  approach  in  DoD,  so  it  is  not  surprising  that  DoD  stakeholders 
would  rank  it  highly  according  to  this  criterion.  Interestingly,  quality  of  care  and  help-seeking 
were  ranked  in  the  lower  half  in  this  category.  Restricting  access  to  lethal  means  was  rated  lowest 
among  all  the  strategies  in  terms  of  cultural  acceptahility.  This  is  also  not  surprising,  given  that 
there  are  signihcant  challenges  associated  with  restricting  access  to  hrearms  among  a  popula¬ 
tion  that  requires  weapons  to  perform  its  mission. 

Cost 

DoD  participants  rated  screening  and  quality  of  care  the  highest  in  terms  of  cost  to  implement 
in  suicide  prevention.  Interestingly,  participants  gatekeeper  training  the  bottom,  despite 
the  potential  costs  associated  with  training  the  entire  force  in  gatekeeper  approaches,  which 
has  been  done  on  “stand-down”  days. 

Learning  Potential 

Quality  of  care  and  continuity  of  care  were  rated  as  having  the  highest  potential  to  inform  future 
suicide  prevention  activities.  Reducing  access  to  lethal  means  was  rated  as  having  the  lowest 
learning  potential,  potentially  because  it  has  already  been  widely  studied  in  other  settings  or 
because  of  the  challenges  identihed  in  terms  of  the  cultural  acceptability  of  this  strategy. 

Additional  Elicitation  from  RAND  Experts 

Because  of  the  lower-than-anticipated  participation  in  the  RAND  ExpertLens  process,  we 
sought  to  confirm  responses  in  two  domains:  effectiveness  and  learning  potential.  We  initi¬ 
ated  a  modihed  expert  elicitation  with  seven  RAND  researchers,  all  of  whom  have  conducted 
recent  research  on  or  related  to  military  suicide  prevention.  Beyond  gathering  additional  data 
that  we  could  use  to  confirm  the  DoD  responses,  this  process  also  allowed  us  to  tap  the  subject- 
matter  expertise  of  RAND  staff  who  conduct  research  supported  by  DoD  related  to  suicide 
prevention,  psychological  health,  and  resilience. 

For  this  second  elicitation,  we  used  the  same  list  of  aspirational  goals  but  asked  RAND 
staff  to  evaluate  each  prevention  strategy  according  to  two  criteria  instead  of  five:  effectiveness 
and  learning  value.  The  evaluation  questions  posed  to  the  RAND  experts  were  edited  slightly 
to  tap  their  specihc  understanding  and  knowledge  of  the  existing  evidence  base  and  research 
literature  related  to  suicide  prevention.  The  wording  for  these  questions  is  shown  in  Table  3.6. 


Table  3.6 

Evaluation  Criteria  Employed  with  RAND  Experts 


Criterion  1:  How  effective  is  this  goal  in  reducing  suicide  attempts  among  those  who  receive  the  stated 
intervention  strategy? 


Not  Very 

effective  effective 

1  23456789  10 

Criterion  2:  Given  your  current  understanding  of  the  research  evidence,  how  important  is  it  to  invest  in  further 
research  on  this  goai? 


Not 

important 

1  2 


Very 

important 

3456789  10 
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The  elicitation  process  for  the  RAND  experts  also  employed  three  rounds  of  data  col¬ 
lection,  but  it  was  implemented  over  email  and  in  person  rather  than  through  the  RAND 
ExpertLens  software  system.  For  the  first  round,  RAND  experts  were  provided  with  two  doc¬ 
uments:  a  compendium  of  research  findings  from  the  literature  on  suicide  prevention  (and 
access  to  a  library  of  the  original  articles)  and  a  rating  form  for  the  12  goals.  RAND  experts 
were  asked  to  review  the  research  findings,  complete  the  rating  forms,  and  submit  responses  by 
email.  Responses  were  the  collated,  tabulated,  and  presented  to  the  group  of  RAND  experts 
during  an  in-person/teleconference  session  connecting  experts  in  different  locations  for  review 
and  discussion  (round  2).  We  used  the  data  collected  from  RAND  experts  on  criterion  1  (effec¬ 
tiveness)  to  rank  the  research  goals,  where  the  goal  with  the  highest  “score”  for  effectiveness 
was  considered  the  top  priority.  RAND  experts  were  asked  to  discuss  and  refine  the  rankings 
of  the  goals  based  on  each  other’s  feedback  (round  3). 

Results 

During  the  discussion,  the  group  of  RAND  experts  chose  to  exclude  one  goal — “implement 
strategies  to  prevent  suicide  reattempts” — from  the  ranking  exercise  because  they  believed  it 
overlapped  or  was  redundant  with  four  other  goals:  psychosocial  interventions,  quality  care, 
provider  training,  and  biological  interventions.  Specifically,  they  felt  that  these  other  four  strat¬ 
egies,  if  implemented  properly,  would  be  the  strategies  used  to  prevent  individuals  who  had 
attempted  suicide  from  attempting  it  again. 

At  the  end  of  each  discussion  of  rankings,  the  RAND  experts  were  asked  to  confirm  the 
group’s  final  ranking/ordering  (round  3).  The  results  of  the  RAND  expert  rankings  in  terms 
of  relative  effectiveness  and  learning  value/investment  importance  are  presented  in  Table  3.7. 
In  comparing  the  RAND  expert  rankings  with  those  generated  through  the  DoD  RAND 
ExpertLens  process,  we  observe  significant  differences  in  opinions  with  respect  to  perceived 


Table  3.7 

RAND  Experts'  Final  Ranking  of  Perceived  Effectiveness  and  Learning  Potential 


Aspirational  Goal 

Effectiveness 

Learning  Potential 

ExpertLens  Rank 

RAND  Rank 

ExpertLens  Rank 

RAND  Rank 

Psychosocial  interventions 

7 

1 

5 

7 

Quality  care 

3 

2 

1 

12 

Reduced  access 

9 

3 

12 

6 

Screening 

12 

4 

10 

8 

Provider  training 

1 

5 

6 

11 

Risk  reduction 

10 

6 

8 

1 

Gatekeeper  training 

4 

7 

4 

2 

Help-seeking 

2 

8 

7 

5 

Continuity  of  care 

5 

9 

2 

10 

Biological  interventions 

11 

10 

11 

4 

Prediction 

8 

11 

3 

9 

Prevent  reattempts 

6 

N/A 

9 

3 
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effectiveness  and  learning  potential.  RAND  experts  were  more  likely  to  rate  strategies  with 
more  existing  research  as  more  effective  than  were  the  DoD  stakeholders.  For  example,  RAND 
experts  rated  reduced  access  to  lethal  means  among  the  top  three  in  terms  of  effective  strategies 
for  reducing  or  preventing  suicide,  hut  it  was  rated  in  the  bottom  third  in  terms  of  effectiveness 
hy  DoD  participants. 

In  terms  of  perceived  learning  potential,  there  was  some  overlap  in  the  rankings  between 
DoD  participants  and  RAND  experts.  For  example,  both  ranked  prediction  strategies  third. 
However,  there  was  divergence  at  the  two  ends  of  the  scale.  DoD  participants  ranked  quality  of 
care  at  the  top  in  terms  of  learning  potential,  whereas  the  RAND  experts  rated  it  at  the  bottom. 
In  fact,  many  of  the  strategies  ranked  as  having  least  learning  potential  by  the  RAND  panel 
were  ranked  as  such  because  the  panel  felt  that  signihcant  research  and  efforts  had  already  been 
devoted  to  these  areas  and  that  there  is  not  much  more  to  learn.  (This  was  voiced  emphatically 
for  enhancing  continuity  of  care  within  DoD.)  Again,  the  RAND  experts  were  instructed  spe¬ 
cifically  to  use  the  existing  literature  to  guide  their  ratings,  whereas  the  DoD  rankings  may  be 
influenced  by  additional  factors  as  well. 

Later  in  this  report,  where  we  use  these  rankings  as  inputs,  we  rely  on  the  RAND  expert 
rankings  because  of  their  connection  to  the  existing  research  base  in  an  effort  to  reduce  the 
noise  that  may  be  associated  with  other  influences. 


Conclusion 

Organizations  within  DoD  that  implement  suicide  prevention  strategies  expect  that  the  activi¬ 
ties  and  interventions  they  choose  to  use  will  have  at  least  a  modest  effect  on  the  outcome  of 
interest.  To  most  effectively  reduce  the  number  and  rate  of  completed  suicides  and  suicide 
attempts,  decisionmakers  need  to  implement  the  strategies  that  have  demonstrated  efficacy 
and  effectiveness.  Those  with  the  most  efficacy  or  effectiveness  are  likely  to  be  high  priorities 
for  implementation."*  As  outlined  earlier  and  in  other  reports,  not  all  strategies,  interventions, 
or  approaches  to  reducing  and  preventing  suicide  have  been  rigorously  tested  or  evaluated,  and 
there  may  not  be  an  evidence  base  to  inform  decisionmakers  about  the  value  or  utility  of  such 
approaches.  And  even  those  that  have  research  support  may  not  reach  100-percent  effective¬ 
ness.  Thus,  research  on  suicide  prevention  and  reduction  strategies  remains  a  critical  need. 
However,  the  priorities  and  goals  for  such  research  may  differ  from  those  for  suicide  prevention 
implementation  strategies.  Instead,  research  goals  may  be  oriented  toward  learning  more  about 
novel  approaches  or  existing  approaches  that  require  adaptation  to  new  settings  or  unique 
populations. 

Decisionmakers  can  decide  to  use  the  rankings  provided  in  this  chapter  to  guide  their 
own  evaluation  of  research  needs  for  military  suicide  prevention.  Ideally,  one  would  want  to 
fund  research  for  strategies  that  DoD  stakeholders  consider  important,  culturally  acceptable, 
and  cost-effective,  in  addition  to  those  that  are  likely  to  be  effective  and  that  have  high  future 


^  As  discussed  in  Chapter  Six,  efficacy  research  typically  involves  small-scale  studies  in  which  participants  have  to  meet 
strict  inclusion  and  exclusion  criteria,  thus  making  such  participants  generally  quite  different  from  the  individuals  in  the 
overall  community  of  interest.  In  effectiveness  research,  studies  come  out  of  the  laboratory  and  into  the  real  world  with  less 
stringent  criteria  for  participation.  Effectiveness  studies  help  test  whether  research  programs  can  be  translated  successfully 
into  real-world  settings  and  whether  the  programs  have  the  components  necessary  to  achieve  such  diffusion. 
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learning  potential.  According  to  these  criteria,  gatekeeper  training  stands  out:  Although  not 
rated  as  currently  very  effective,  it  was  ranked  hy  DoD  representatives  as  important,  cultur¬ 
ally  acceptable,  and  least  costly;  the  RAND  panel  also  ranked  it  as  having  the  second  high¬ 
est  learning  potential.  Furthermore,  prior  research  suggests  that  these  types  of  trainings  are 
widely  used  in  DoD  (Ramchand  et  ah,  2011).  Provider  training  and  strategies  for  enhancing 
continuity  of  care  appear  promising  in  terms  of  their  importance  and  effectiveness;  however, 
the  RAND  panelists  ranked  these  strategies  as  likely  to  yield  low  returns  in  future  research, 
largely  because  they  believed  that  there  is  not  much  more  to  learn  about  these  approaches. 
Then  again,  although  the  RAND  panel  thought  there  was  much  to  learn  about  potential  bio¬ 
logical  approaches,  it  was  ranked  by  DoD  representatives  as  being  unimportant,  not  culturally 
acceptable,  and  costly.  It  may  also  be  helpful  to  aggregate  these  rankings  systematically,  which 
we  do  in  Chapter  Five. 

The  collection  of  perspectives  and  insights  from  relevant  DoD  stakeholders  and  RAND 
experts  yielded  important  considerations  for  a  strategic  research  agenda  for  DoD.  The  process 
is  not  without  its  limitations,  however.  Of  primary  concern  is  the  low  response  rate  from  DoD 
stakeholders  in  the  online  RAND  ExpertLens  elicitation.  Further,  the  results  may  be  biased 
on  the  perspective  of  the  respondents  who  did  participate  (the  majority  were  “implementers” 
as  opposed  to  “researchers”).  Future  efforts  within  DoD  could  use  a  process  similar  to  that 
described  here  and  augment  it  with  incentives  to  encourage  participation  to  ensure  that  the 
results  represent  a  broader  panel.  Despite  communicating  the  objective  and  nature  of  this 
independent  elicitation  process  and  the  expressed  goal  of  gathering  individuals’  subject-matter 
expertise  (rather  than  official  input  as  an  office  representative),  we  believe  the  low  uptake  rate 
may  be  associated  with  concerns  about  participating  without  having  been  tasked  by  leadership 
to  do  so.  DoD  leadership  is  ideally  situated  to  confront  this  challenge. 

Even  with  the  low  response  rate,  the  results  of  the  process  still  provide  important  insights 
from  DoD  stakeholders  with  respect  to  priorities  and  aspirational  goals.  They  should  not  be 
interpreted  as  representative  of  all  DoD  organizations,  branches  of  service,  or  defense  agen¬ 
cies;  rather,  they  should  be  viewed  as  helpful  insight  and  input  from  professionals  invested  in 
DoD  suicide  prevention  efforts.  Each  of  the  experts  who  participated  is  actively  engaged 
in  studies  designed  to  assist  and  improve  DoD’s  policies,  strategies,  resources,  and  interven¬ 
tions  designed  to  reduce  suicide  and  improve  psychological  health  and  resilience  among  uni¬ 
formed  service  members  and  their  families.  Their  perspectives  should,  in  some  way,  influence 
how  DoD  spends  its  research  funds. 


CHAPTER  FOUR 


Preliminary  Gap  Analysis 


At  this  point,  we  have  presented  ahstractions  of  all  DoD  and  non-DoD  research  on  suicide 
prevention  (Chapter  Two)  and  identihed  DoD  research  needs  and  priorities  (Chapter  Three). 
In  this  chapter,  we  examine  whether  current  research  funding  aligns  with  DoD’s  research 
priorities. 


Methods 

We  present  two  stacked  har  charts  for  each  of  five  domains:  perceived  importance,  effective¬ 
ness,  cultural  acceptability,  cost,  and  future  learning  potential.  In  each  set,  the  first  graph 
shows  the  number  of  studies  funded  for  each  aspirational  research  goal,  separating  those  con¬ 
ducted  within  and  outside  DoD,  with  higher-ranked  priorities  at  the  top  and  lower-ranked 
priorities  at  the  bottom.  We  then  present  the  same  information  in  the  same  manner  for  the 
amount  of  funding  currently  allocated  to  each  research  goal.  This  visualization  allows  us  to  see 
in  a  basic  way  whether  current  studies  align  with  DoD’s  research  priorities. 

As  described  in  Chapter  Three  (see  Table  3.2),  we  modified  the  goals  we  used  for  the 
study  abstraction  to  the  ones  we  used  for  the  RAND  ExpertLens  solicitation.  To  recap, 
the  original  NAASP  goal  “risk  and  protective  factor  interactions”  was  not  included  in  the 
RAND  ExpertLens  process.  Thus,  the  funding  allocated  to  such  studies  is  not  included  in 
our  analysis.  However,  that  exclusion  did  not  limit  our  ability  to  conduct  our  analysis,  which 
involved  analyzing  how  those  categories  ranked  by  the  experts  are  funded  relative  to  one 
another.  Chapter  Two  accounted  for  this  funding  in  presenting  the  total  amount  funded. 

In  addition,  the  original  NAASP  goal  “provider  and  gatekeeper  training”  was  broken 
into  two  goals:  (1)  ensure  that  non-health  professionals  (i.e.,  noncommissioned  officers,  chap¬ 
lains)  who  come  into  contact  with  suicidal  individuals  are  trained  to  identify,  care  for,  and 
refer  persons  at  risk,  and  (2)  train  health  care  professionals  to  identify  those  at  risk  for  suicide 
and  to  manage  their  treatment.  To  accommodate  these  changes,  we  reviewed  the  23  studies 
originally  identified  as  provider/gatekeeper  studies  (see  Appendix  A)  to  isolate  those  that  were 
provider  studies  (number  funded  by  DoD  =  7;  DoD  funding  =  $19,389,657;  number  funded 
outside  DoD  =  9;  non-DoD  funding  =  $4,714,538)  and  those  that  were  gatekeeper  studies 
(number  funded  by  DoD  =  4;  DoD  funding  =  $13,664,667;  number  funded  outside  DoD  =  6; 
non-DoD  funding  =  $3,433,442).  Of  the  23  studies  we  reviewed,  we  identified  three  (two 
funded  by  DoD  and  one  funded  by  a  non-DoD  entity)  as  both  provider  and  gatekeeper  stud¬ 
ies,  and  these  studies  are  included  in  both  totals. 
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Results 

Perceived  Importance 

The  aspirational  goals  with  the  largest  number  of  studies,  both  in  aggregate  and  when  examin¬ 
ing  DoD  funding  specifically,  are  studies  on  quality  of  care  and  psychosocial  interventions,  which 
DoD  panelists  ranked  as  fifth  and  ninth,  respectively,  in  perceived  importance  (see  Figure  4.1). 
There  are  fewer  than  20  studies  on  provider  training,  which  was  ranked  as  the  most  important 
goal,  and  only  ten  on  gatekeeper  training,  which  was  ranked  as  the  third  most  important. 

As  was  true  for  the  number  of  studies,  the  largest  amount  of  funding  is  being  spent  on 
studies  of  quality  care  (particularly  by  DoD)  and  psychosocial  interventions  (see  Figure  4.2). 
However,  there  is  also  more  than  $40  million  being  spent  on  research  focused  on  enhanc¬ 
ing  continuity  of  care,  and  DoD  is  contributing  more  than  half  of  these  funds.  DoD  panelists 
ranked  this  strategy  as  the  second  most  important  for  suicide  prevention. 

Effectiveness 

The  two  areas  that  the  RAND  experts  ranked  as  currently  most  effective — psychosocial 
interventions  and  quality  care — are  the  same  goals  with  the  largest  number  of  studies  (see 
Figure  4.3).  However,  the  third  most  effective — reduced  access  to  lethal  means — had  very  little 
funding,  both  by  DoD  and  in  aggregate.  It  is  also  noteworthy  that  there  were  a  significant 
number  of  studies  on  enhancing  continuity  of  care,  biological  interventions,  and  prediction  of 
imminent  risk — areas  that  were  ranked  among  the  lowest  according  to  their  current  evidence 
of  effectiveness. 

An  examination  of  the  amount  of  funding  reveals  both  parallels  and  differences  (see 
Figure  4.4).  First,  most  funding  is  going  to  the  two  areas  with  the  largest  number  of  studies 
and  that  were  ranked  first  and  second  in  terms  of  effectiveness — psychosocial  interventions  and 
quality  care — whereas  the  least  amount  of  funding  is  currently  dedicated  to  what  was  ranked 
the  third  most  effective  strategy:  reduced  access  to  lethal  means.  Screening,  which  was  ranked  as 


Figure  4.1 
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Figure  4.2 

Amount  of  Funding,  by  Perceived  Importance 
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Figure  4.3 

Number  of  Studies,  by  Effectiveness 
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fourth  most  effective,  had  similarly  low  levels  of  funding.  The  funding  allocated  to  the  rest  of 
the  goals  generally  correlated  with  their  effectiveness  rankings. 


Cultural  Acceptability 

When  plotted  hy  ranking  of  cultural  acceptability,  the  number  of  studies  almost  produces  a 
bell-shaped  curve,  with  the  fewest  studies  on  both  the  most  and  least  culturally  acceptable 
strategies  and  the  most  studies  being  funded  in  the  categories  that  were  ranked  somewhere 
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Figure  4.4 

Amount  of  Funding,  by  Effectiveness 
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in  the  middle  (see  Figure  4.5).  The  same  pattern  generally  holds  when  examining  the  total 
amount  funded  hy  ranking  of  cultural  acceptahility  (see  Figure  4.6). 

Cost 

With  screening  and  psychosocial  interventions  as  outliers,  the  aggregate  number  of  studies  being 
conducted  across  the  aspirational  goals  correlates  with  the  cost  of  implementing  the  given  strat¬ 
egy,  with  more  studies  being  funded  on  strategies  that  were  ranked  as  more  costly  to  imple- 


Figure  4.5 

Number  of  Studies,  by  Cultural  Acceptability 
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Figure  4.6 

Amount  of  Funding,  by  Cultural  Acceptability 
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ment  (see  Figure  4.7).  Again,  screening  stands  out  as  being  ranked  as  the  most  costly  to  imple¬ 
ment,  but  there  are  also  few  studies  being  funded  on  screening.  While  DoD  funding  patterns 
parallel  the  overall  results,  the  category  of  biological  interventions  also  stands  out  as  an  outlier, 
with  very  few  DoD  studies  focused  on  this  type  of  intervention;  it  was  also  ranked  as  the  third 
most  costly  strategy  to  implement.  Again,  these  same  patterns  are  apparent  when  the  amount 
being  spent  on  research  studies  is  ranked  in  order  of  the  cost  to  implement  the  strategies  (see 
Figure  4.8). 


Figure  4.7 

Number  of  Studies,  by  Cost  to  Implement 
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Figure  4.8 

Amount  of  Funding,  by  Cost  to  Implement 
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Learning  Potential 

Finally,  when  ordered  with  respect  to  learning  potential,  according  to  the  RAND  experts,  we 
see  that  the  largest  numher  of  studies  and  the  most  research  funding — both  hy  DoD  and  in 
aggregate — address  strategies  ranked  as  having  the  lowest  learning  potential  (see  Figure  4.9). 
The  exceptions  are  that  there  are  few  studies  (with  little  funding)  on  screening,  which  was 
ranked  eighth,  and  there  is  a  relatively  large  numher  of  studies  hut  correspondingly  little  fund¬ 
ing  going  toward  biological  interventions,  which  was  ranked  as  having  the  fourth  highest  learn¬ 
ing  potential  (see  Figure  4.10).  Recall  that  most  studies  in  this  area  are  funded  hy  the  AFSR 

Figure  4.9 

Number  of  Studies,  by  Learning  Potential 
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Figure  4.10 

Amount  of  Funding,  by  Learning  Potential 
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Conclusion 

We  combined  the  data  from  Chapters  Two  and  Three  to  produce  a  preliminary  gap  analy¬ 
sis  examining  whether  current  research  on  suicide  prevention  with  relevance  to  military  per¬ 
sonnel  aligns — in  terms  of  both  the  number  of  ongoing  studies  and  the  amount  of  funds 
allocated — with  DoD’s  research  needs.  We  examined  these  needs  across  five  domains:  impor¬ 
tance,  effectiveness,  cultural  acceptability,  cost,  and  learning  potential.  The  results  suggest  that 
current  studies  and  funding  align  best  with  the  domains  of  effectiveness  and  cost.  In  other 
words,  there  are  more  studies  and  more  funding  going  toward  suicide  prevention  strategies 
that  are  already  ranked  as  highly  effective,  as  well  as  those  ranked  as  the  most  costly  to  imple¬ 
ment.  However,  there  is  no  apparent  relationship  between  what  is  being  funded  and  what 
DoD  representatives  perceive  as  important.  Furthermore,  while  there  are  few  studies  and 
little  funding  being  allocated  to  strategies  ranked  as  least  culturally  acceptable,  there  are  also 
few  studies  and  little  funding  allocated  to  strategies  ranked  as  most  culturally  acceptable. 
Finally,  there  is  an  inverse  relationship  between  the  number  of  studies  and  amount  of  fund¬ 
ing  being  devoted  to  strategies  with  the  most  learning  potential,  with  more  studies  and 
funding  going  toward  strategies  ranked  as  having  the  lowest  learning  potential. 


CHAPTER  FIVE 

Modeling  DoD's  Suicide  Prevention  Research  Priorities 


In  this  chapter,  we  use  data  from  both  the  RAND  ExpertLens  ranking  with  DoD  and  the 
RAND  expert  panel,  described  in  Chapter  Three,  to  provide  a  practical  framework  for  pri¬ 
oritizing  aspirational  research  goals  to  yield  maximum  impact  in  terms  of  reduced  suicide 
attempts  and  cost.  This  framework  accounts  for  differing  population  subgroups  of  military 
personnel  who  are  affected  by  each  research  strategy.  It  models  the  effect  and  costs  of  interven¬ 
tions  matched  to  these  subgroups.  In  addition,  we  draw  insights  from  the  economics  literature 
on  research  and  development  (R&D)  portfolio  optimization  to  provide  guidance  for  decision¬ 
makers  about  allocating  DoD  funds  for  research  on  military  suicide  prevention. 


Background 

The  foundation  for  the  analysis  in  this  chapter  comes  from  research  conducted  by  the  NAASP 
Research  Prioritization  Task  Force  (Pringle  et  ah,  2013).  In  that  work,  the  authors  highlight  a 
fundamental  trade-off  between  the  breadth  (i.e.,  the  size  of  the  population  affected)  and  depth 
(i.e.,  the  effectiveness)  of  suicide  interventions  targeted  at  various  subgroups.  For  illustrative 
purposes,  the  authors  predefine  three  subgroups  that  vary  in  size  (large,  medium,  and  small). 
They  then  extract  specihc  statistics  for  each  subgroup,  such  as  proportion  with  suicidal  ide¬ 
ation,  proportion  with  serious  mental  illness,  and  proportion  that  has  attempted  suicide.  In  the 
process,  they  note  that  the  broadest  subgroup  possesses  a  relatively  low  rate  of  suicide  attempts 
compared  with  a  narrow  subgroup.  Incorporating  additional  targeting  factors  (i.e.,  further 
dividing  subgroups)  generally  increases  the  intervention  resources  dedicated  per  person  at  the 
expense  of  narrowing  the  scope  and  excluding  at-risk  populations.  Thus,  the  optimal  preven¬ 
tion  intervention  espoused  by  the  authors  would  balance  the  breadth  and  depth  of  interven¬ 
tions  to  maximize  overall  suicide  prevention  (i.e.,  the  total  number  of  suicides  prevented). 

Just  as  Pringle  and  colleagues  (2013)  focus  on  the  targeted  delivery  of  an  intervention,  in 
a  similar  fashion,  research  strategies  can  be  characterized  as  affecting  “universal,”  “selected,”  or 
“indicated”  subgroups.  For  example,  research  on  population  risk  reduction,  gatekeeper  train¬ 
ing,  stigma  reduction,  screening,  and  prediction  universally  affects  the  entire  population,  while 
research  on  provider  training  and  continuity  of  care  is  more  targeted  and  affects  only  selected 
subgroups  at  increased  risk  (i.e.,  those  in  contact  with  a  health  care  provider).  An  indicated 
subgroup  is  one  with  a  past  history  of  mental  illness  or  suicide  attempts.  In  the  methodology 
section,  we  explore  in  greater  detail  the  assumptions  made  in  determining  the  size  of  each 
subgroup  of  military  personnel.  Nevertheless,  the  same  trade-off  between  breadth  and  depth 
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exists  for  prioritizing  research  strategies,  because  different  subgroups  display  different  degrees 
of  suicide  risk. 


Methods 

In  this  section,  we  model  the  impact  of  the  full  implementation  of  interventions  that  corre¬ 
spond  to  the  research  strategy  for  the  appropriate  subgroups.  The  goal  is  to  rank  prevention 
interventions  with  a  metric  that  accounts  for  both  the  explicit  benefits  (suicides  prevented)  and 
the  explicit  costs  (implementation  costs). 

Model  Parameters 

An  ideal  intervention  would  result  in  a  large  benefit  for  the  least  cost  (i.e.,  a  big  “bang  for  the 
buck”).  To  formalize  this  concept,  we  developed  a  benefit-cost  index  based  on  parameters  that 
may  be  unique  to  each  research  strategy: 

•  n  =  total  population  size. 

•  p  =  proportion  (percentage)  of  total  population  affected  by  research  strategy. 

•  r  =  suicide  attempt  rate  for  the  subgroup,  in  terms  of  per-person  risk  per  year  (percentage). 

•  i  =  intervention  effectiveness  rate  (percentage  of  suicide  attempts  averted). 

•  c  =  total  cost  of  treatment  for  the  subgroup. 

The  total  population  size  {n)  in  this  setting  is  the  total  active-duty  military  population 
of  approximately  1.4  million  people  (Office  of  the  Deputy  Under  Secretary  of  Defense  for 
Military  Community  and  Family  Policy,  2012).  As  discussed  earlier,  each  research  strategy 
is  assumed  to  affect  a  different  proportion  of  the  total  population  {p) — universal,  selected,  or 
indicated  subgroups.  The  parameter  p  captures  the  idea  of  “breadth,”  but  having  a  broad  reach 
is  not  meaningful  unless  the  subgroup  affected  is  actually  at  risk  of  attempting  suicide.  The  per¬ 
centage  of  the  subgroup  at  risk  of  attempting  suicide  (r)  is  expressed  in  terms  of  risk  per  person 
per  year  and  represents  the  “depth”  of  the  subgroup.  Certain  interventions  may  affect  a  small 
group  of  people  who  are  at  a  very  high  risk  of  attempting  suicide,  while  other  interventions 
affect  a  large  number  people,  each  with  a  low  risk.  The  product  of  the  total  population  size  {ri), 
the  proportion  of  the  total  population  affected  {p),  and  the  suicide  attempt  rate  (r)  gives  us  the 
number  of  people  attempting  suicide  each  year.  Multiplying  the  number  of  people  attempting 
suicide  by  the  intervention  effectiveness  rate  (z)  (i.e.,  the  percentage  of  suicide  attempts  averted) 
gives  us  the  explicit  benefit  from  pursuing  each  research  strategy  and  serves  as  the  numerator 
for  our  index.  The  denominator  consists  of  the  total  per-person  cost  of  treatment,  so  the  hnal 
bang-for-the-buck  index  is  expressed  in  terms  of  suicides  prevented  per  dollar  spent. 

For  each  research  strategy,  the  following  benefit-cost  index  can  be  calculated: 

suicides  prevented  n'X  pXsXi 

index  =  - = - . 

cost  c 
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Parameterization  of  the  Model 

We  used  a  combination  of  data  gleaned  from  the  RAND  ExpertLens  and  RAND  expert 
elicitation  panels,  as  well  as  previous  research,  to  assign  values  to  the  parameter  for  each  data 
strategy. 

Proportion  of  Military  Population  Affected 

Table  5.1  shows  the  assumptions  that  directly  underlie  the  values  for  the  proportion  of  the  mili¬ 
tary  population  affected  (p)  by  each  aspirational  goal.  For  universal  approaches  (risk  reduction, 
gatekeeper  training,  help-seeking,  screening,  reduced  access  to  lethal  means,  and  prediction 
modeling),  we  assume  that  the  entire  population  of  military  personnel  is  affected.  For  health 
service  delivery  interventions  not  specifically  focused  on  those  with  identified  mental 
health  problems,  we  assume  that  all  those  meeting  symptom  criteria  of  having  a  probable 
mental  health  problem  are  affected,  regardless  of  whether  they  have  a  mental  health  diagnosis 
or  are  seeking  mental  health  treatment,  and  we  estimate  that  25  percent  of  military  person¬ 
nel  have  a  mental  illness  (Riddle  et  ak,  2007).  Finally,  for  interventions  focused  specifically 
on  individuals  with  mental  illness,  we  assume  the  median  between  those  diagnosed  with  a 
mental  illness  (9  percent;  Armed  Forces  Health  Surveillance  Center,  2012)  and  those  with 
a  mental  illness,  including  those  without  having  ever  received  a  diagnosis,  or  17  percent;  for 
prevention  of  reattempts,  we  estimate  that  2  percent  of  military  personnel  report  attempting 
suicide  in  the  past  year  (Bray  et  ak,  2009). 


Table  5.1 

Proportion  of  Military  Population  Affected 


Aspirational  Goal 

%  of  Force 
Affected 

Basis  for  Proportion  of  Force  Affected 

Risk  reduction 

100 

Assume  all  are  affected 

Gatekeeper  training 

100 

Assume  all  are  affected 

Help-seeking 

100 

Assume  all  are  affected 

Screening 

100 

Assume  all  are  affected 

Reduced  access 

100 

Assume  all  are  affected 

Prediction 

100 

Assume  all  are  affected 

Provider  training 

25 

Assume  all  military  personnel  access  care  and  those  with  mental 
illness  are  recognized 

Continuity  of  care 

25 

Assume  all  military  personnel  access  care  and  those  with  mental 
illness  are  recognized 

Psychosocial  interventions 

17 

Assume  median  of  military  personnel  diagnosed  with  a  mental 
illness  (9%)  and  those  meeting  symptom  criteria  of  having  a 
probable  mental  illness,  including  those  who  may  have  never  been 
diagnosed  or  in  mental  health  treatment  (25%) 

Biological  interventions 

17 

Assume  median  of  military  personnel  diagnosed  with  a  mental 
illness  (9%)  and  those  with  a  mental  illness  (25%) 

Quality  care 

17 

Assume  median  of  military  personnel  diagnosed  with  a  mental 
illness  (9%)  and  those  with  a  mental  illness  (25%) 

Prevent  reattempts 

2 

2  percent  of  active-duty  personnel  report  having  attempted  suicide 
in  the  prior  year 
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Proportion  of  Subgroup  at  Risk  of  Dying  by  Suicide 

Table  5.2  shows  our  assumptions  that  directly  underlie  the  values  for  the  percentage  of  the 
subgroup  (p)  at  risk  of  dying  by  suicide  (s)  in  terms  of  pet-person  risk  per  year.  Again,  for  uni¬ 
versal  approaches,  we  assume  that  individual  risk  is  0.02  percent  (derived  from  the  fact  that 
the  annual  suicide  rate  in  DoD  has  hovered  around  two  per  10,000).  For  any  interventions 
delivered  to  personnel  with  mental  illness,  we  assume  that  the  risk  is  0.40  percent,  based  on 
previous  research  suggesting  that  the  risk  of  suicide  for  those  with  mental  health  problems  is 
20  times  that  of  the  general  population;  for  interventions  targeting  those  who  have  attempted 
suicide,  we  assume  that  the  risk  is  0.90  percent,  based  on  previous  research  suggesting  that  the 
risk  of  suicide  among  attempters  is  40-50  times  that  of  the  general  population  (Harris  and 
Barraclough,  1997). 

Intervention  Effectiveness 

For  our  intervention  effectiveness  parameter,  we  did  not  feel  confident  in  even  making  rough 
estimates  from  the  literature.  Instead,  we  use  the  results  from  the  RAND  expert  panel.  As 
described  in  Chapter  Three,  we  convened  an  internal  expert  panel  to  rank  the  research  strate¬ 
gies  in  terms  of  relative  effectiveness  (i.e.,  the  percentage  of  suicide  attempts  in  the  subgroup 
that  would  be  averted  because  of  treatment).  The  second  step  involved  assigning  bounds  to  the 
final  ranking  (i.e.,  asking  the  experts  to  agree  on  actual  percentage  values  for  the  least  effec¬ 
tive  rate  and  for  the  most  effective  rate).  We  combined  the  rankings  and  the  bounds  to  create 


Table  5.2 

Proportion  of  Subgroup  at  Risk  of  Attempting  Suicide 


Aspirational  Goal 

Risk  per 
Person  per 
Year (%) 

Basis  for  per-Person  Risk  per  Year 

Risk  reduction 

0.02 

Current  risk  in  DoD  (2  per  10,0000) 

Gatekeeper  training 

0.02 

Current  risk  in  DoD  (2  per  10,0000) 

Help-seeking 

0.02 

Current  risk  in  DoD  (2  per  10,0000) 

Screening 

0.02 

Current  risk  in  DoD  (2  per  10,0000) 

Reduced  access 

0.02 

Current  risk  in  DoD  (2  per  10,0000) 

Prediction 

0.02 

Current  risk  in  DoD  (2  per  10,0000) 

Psychosocial  interventions 

0.40 

Risk  of  those  with  a  mental  health  problem  is  20  times  greater 
(i.e.,  0.02%  X  20  =  0.40%) 

Biological  interventions 

0.40 

Risk  of  those  with  a  mental  health  problem  is  20  times  greater 
(i.e.,  0.02%  X  20  =  0.40%) 

Provider  training 

0.40 

Risk  of  those  with  a  mental  health  problem  is  20  times  greater 
(i.e.,  0.02%  X  20  =  0.40%) 

Continuity  of  care 

0.40 

Risk  of  those  with  a  mental  health  problem  is  20  times  greater 
(i.e.,  0.02%  X  20  =  0.40%) 

Quality  care 

0.40 

Risk  of  those  with  a  mental  health  problem  is  20  times  greater 
(i.e.,  0.02%  X  20  =  0.40%) 

Prevent  reattempts 

0.90 

Risk  of  those  with  a  prior  suicide  attempt  is  40-50  times  greater 
(i.e.,  0.02%  X  45  =  0.90%) 
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rough  estimates  for  the  values  of  the  intervention  effectiveness  rate  hy  assuming  a  simple  linear 
interpolation  for  each  research  goal  between  the  minimum  and  maximum  ranking.'  Recall, 
however,  that  the  RAND  expert  panel  did  not  rank  the  prevent  reattempts  goal  because  of 
the  perceived  overlap  with  other  goals  (specifically,  psychosocial  interventions,  quality  care, 
provider  training,  and  biological  interventions).  Thus,  for  the  effectiveness  of  interventions  to 
prevent  reattempts,  we  calculated  the  average  across  these  four  categories.  The  results  are  pre¬ 
sented  in  Table  5.3. 

Individual  Cost  of  Interventions 

What  we  provide  for  the  per-person  cost  of  an  intervention  is  perhaps  the  strongest  set  of 
assumptions.  Based  on  our  assumptions,  per-person  costs  range  from  $1  (our  estimated  cost 
of  an  anti-stigma  campaign  that  encourages  help-seeking)  to  $3,000  (our  estimated  cost  of 
mental  health  treatment)  per  person.  Our  cost  estimates  and  the  basis  for  these  estimates  are 
provided  in  Table  5.4.^ 


Table  5.3 

Intervention  Effectiveness 


Effectiveness 

Aspirational  Goal 

Rank 

Intervention 
Effectiveness  (%) 

Psychosocial  interventions 

1 

55 

Quality  care 

2 

50 

Reduced  access 

3 

45 

Screening 

4 

40 

Prevent  reattempts 

NA 

37.5 

Provider  training 

5 

35 

Risk  reduction 

6 

30 

Gatekeeper  training 

7 

25 

Help-seeking 

8 

20 

Continuity  of  care 

9 

15 

Biological  interventions 

10 

10 

Prediction 

11 

5 

NOTE:  We  did  not  rank  efforts  to  prevent  reattempts  because  of  perceived 
overlap  with  other  goals.  Instead,  we  averaged  the  assumed  effectiveness 
for  psychosocial  interventions,  quality  care,  provider  training,  and  biological 
interventions. 


*  A  more  sophisticated  transformation  of  rankings  to  values  would  have  required  either  more  data  points  or  more  assump¬ 
tions  about  the  shape  of  the  distribution  of  the  effectiveness  rates  for  each  strategy.  Given  the  uncertainty,  linear  interpola¬ 
tion  is  perhaps  the  most  neutral  method  for  generating  the  desired  data  elements. 

^  The  Defense  Suicide  Prevention  Office  has  informed  RAND  that  it  has  a  rigorous  effort  under  way  to  estimate  the  costs 
associated  with  these  types  of  interventions.  When  this  effort  is  complete,  the  data  could  be  used  to  parameterize  the  model 
in  place  of  the  parameters  used  here. 
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Table  5.4 
Per-Person  Cost 


Aspirational  Goal 

Per-Person 

Cost 

Basis  for  Per-Person  Cost 

Help-seeking 

$1 

Expenditure  on  anti-stigma  marketing  campaigns  in  England  2009-2011 
was  $1,129,667  divided  by  total  force  (Evans-Lacko  et  al.,  2013) 

Provider  training 

$15 

Per-provider  cost  of  $115  for  the  Suicide  Prevention  Resource  Center's 
workshop  "Assessing  and  Managing  Suicide  Risk:  Core  Competencies 
for  Mental  Health  Professionals,"  multiplied  by  183,542  mental  health 
professionals  (119,180  active-duty,  42,315  civilian,  802  foreign  national 
civilian,  and  21,245  reserve)  divided  by  total  force  (DMDC,  2012) 

Gatekeeper  training 

$30 

Cost  for  QPR  (Question,  Persuade  and  Refer)  Institute's  online,  two-hour 
training  (Louisiana  Spirit  Suicide  Prevention  Summit  Meeting  Planning 
Committee,  2006) 

Reduced  access 

$50 

Average  of  the  following  mean  restriction  options: 

•  Trigger  lock:  $15 

•  Plastic/fiberglass  gun  case:  $25 

•  Strong  box/metal  gun  case:  $100 

•  Gun  safe:  $125 

•  Locking  steel  gun  cabinet:  $400 

Risk  reduction 

$90 

Army  expenditure  of  $125  million  to  develop  Comprehensive  Soldier  and 
Family  Fitness  program,  divided  by  total  force  (Murphy,  2011) 

Screening 

$200 

Cost  of  a  school-based  mental  health  screening  program,  which  ranged 
from  $149  to  $234  per  student  (Chatterji  et  al.,  2004) 

Prediction 

$200 

No  basis  to  estimate;  assume  equivalent  to  screening 

Continuity  of  care 

$1,000 

Per-patient  cost  of  hospital-based  team  home  care  when  implemented  in 
the  VA  in  1985  (Hughes  et  al.,  1992) 

Biological  interventions 

$3,000 

Evidence-based  treatment  for  comorbid  PTSD  and  major  depression  = 
$2,989.13  (Eibner  et  al.,  2008;  Foa,  Davidson,  and  Frances,  1999) 

Prevent  reattempts 

$3,000 

Evidence-based  treatment  for  comorbid  PTSD  and  major  depression  = 
$2,989.13  (Eibner  et  al.,  2008;  Foa,  Davidson,  and  Frances,  1999) 

Psychosocial  interventions 

$3,000 

Evidence-based  treatment  for  comorbid  PTSD  and  major  depression  = 
$2,989.13  (Eibner  et  al.,  2008;  Foa,  Davidson,  and  Frances,  1999) 

Quality  care 

$3,000 

Evidence-based  treatment  for  comorbid  PTSD  and  major  depression  = 
$2,989.13  (Eibner  et  al.,  2008;  Foa,  Davidson,  and  Frances,  1999) 

Analysis 

For  each  aspirational  goal,  we  first  calculated  the  henefit-cost  index.  We  can  use  this  informa¬ 
tion  to  rank  aspirational  research  goals  in  a  way  that  accounts  for  effectiveness  and  cost  and 
compare  it  to  the  rankings  of  “importance”  derived  from  the  RAND  ExpertLens  panel.  We 
next  took  the  index  value  rankings  and  plotted  them  against  the  cultural  acceptability  rank¬ 
ings,  allowing  us  to  highlight  certain  strategies  that  look  very  promising  based  on  the  explicit 
benefit-cost  index  but  that  carry  high  implicit  implementation  costs  (i.e.,  ranked  low  with 
respect  to  cultural  acceptability)  and  that,  as  a  consequence,  would  need  to  be  discounted. 
Finally,  we  drew  on  recent  insights  from  the  economics  literature  on  R&D  portfolio  choice 
and  learning  value  to  incorporate  both  the  benefit-cost  index  and  future  learning  potential, 
shedding  further  insight  on  optimizing  a  suicide  prevention  research  portfolio. 
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Results 

Suicides  Prevented  per  Cost 

The  intent  of  benefit-cost  ratio  analysis  is  to  break  down  the  problem  of  prioritizing  research 
goals  into  discrete  factors.  Decisions  about  which  goals  to  pursue  must  take  into  account  all 
four  factors:  the  size  of  the  subgroup,  the  subgroup’s  suicide  risk,  the  intervention  effectiveness 
rate,  and  the  cost  of  treating  the  subgroup.  Upon  calculating  the  benefit-cost  index  for  each 
research  strategy,  we  arrive  at  a  ranking  of  strategies  to  yield  a  maximum  impact  in  terms  of 
reduced  morbidity  (suicide  attempts),  given  the  cost  of  implementation.  These  results  are  pre¬ 
sented  in  Table  5.5  and  ranged  from  a  high  of  93.3  for  provider  training  to  a  low  of  0.05  for 
prediction. 

Suicides  Prevented  per  Cost  Relative  to  Overall  Importance 

As  part  of  the  RAND  ExpertLens  protocol,  we  asked  participants  about  the  “overall  impor¬ 
tance”  of  a  certain  strategy  in  lowering  the  suicide  rate  of  military  personnel.  Table  5.6  displays 
rankings  from  this  question  relative  to  the  benefit-cost  index.  In  both  cases,  provider  training 
ranked  first;  other  similarities  are  that  help-seeking  and  gatekeeper  training  both  ranked  toward 
the  top  of  the  list,  while  screening  and  biological  therapies  ranked  toward  the  bottom.  Major 
differences  between  the  approaches  are  that  reduced  access  ranked  toward  the  top  of  the  benefit- 
cost  index  but  toward  the  bottom  with  respect  to  perceived  overall  importance,  and  continuity 
of  care  ranked  second  most  important  for  suicide  prevention  but  fairly  low  in  the  benefit-cost 
index. 


Table  5.5 

Suicides  Prevented  per  Cost  Index 


Aspirational  Goal 

Suicides  Prevented  per 
Cost  Index  Value 

Provider  training 

93.30 

Help-seeking 

49.60 

Reduced  access 

1.80 

Gatekeeper  training 

1.67 

Prevent  reattempts 

1.13 

Psychosocial  interventions 

0.73 

Risk  reduction 

0.67 

Quality  care 

0.67 

Continuity  of  care 

0.60 

Screening 

0.40 

Biological  interventions 

0.13 

Prediction 

0.05 
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Table  5.6 

Benefit-Cost  Index  and  Overall  Importance  Rankings 


Benefit-Cost  Index 

"Overall"  Importance  from 

RAND  ExpertLens  Panel 

1.  Provider  training 

1.  Provider  training 

2.  Help-seeking 

2.  Continuity  of  care 

3.  Reduced  access 

3.  Gatekeeper  training 

4.  Gatekeeper  training 

4.  Help-seeking 

5.  Prevent  reattempts 

5.  Quality  care 

6.  Psychosocial  interventions 

6.  Prediction 

7.  Risk  reduction 

7.  Prevent  reattempts 

8.  Quality  care 

8.  Risk  reduction 

9.  Continuity  of  care 

9.  Psychosocial  interventions 

10.  Screening 

10.  Reduced  access 

11.  Biological  interventions 

11.  Screening 

12.  Prediction 

12.  Biological  interventions 

Benefit-Cost  Index  Relative  to  Cultural  Acceptability 

Figure  5.1  shows  the  trade-offs  between  the  heneht-cost  index  and  cultural  acceptability,  as 
measured  by  participants  in  the  ExpertLens  panel.  As  mentioned  earlier,  this  trade-ofF  space 
highlights  certain  strategies  that  look  very  promising  based  on  the  explicit  benefit-cost  index 
but  carry  high  implicit  implementation  costs  and,  as  a  consequence,  would  need  to  be  dis¬ 
counted  (bottom  right  corner,  e.g.,  reducing  access  to  lethal  means).  Conversely,  strategies  with 
a  lower  benefit-cost  index  but  high  levels  of  cultural  acceptability  might  be  prioritized  higher. 
Strategies  in  the  upper  right  portion  of  the  figure  represent  research  strategies  with  the  most 
promise  in  terms  of  suicides  prevented  per  cost  and  are  most  culturally  acceptable:  provider 
training  zsidi  gatekeeper  training.  Prediction,  screening,  and  biological  interventions  rank  low  with 
respect  to  their  benefit-cost  index  and  cultural  acceptability.  While  continuity  of  care  ranks 
highly  with  respect  to  cultural  acceptability,  it  has  a  relatively  modest  benefit-cost  index. 

Benefit-Cost  Index  Relative  to  Learning  Value 

So  far,  we  have  only  discussed  prioritizing  research  strategies  via  rankings  based  on  benefit-cost 
indexes.  While  these  rankings  inform  the  relative  returns  on  investment  from  funding  suicide 
research  in  1 1  different  areas,  the  question  remains:  Given  a  total  research  budget,  what  is  the 
optimal  allocation  to  each  of  these  research  goals  in  an  R&D  portfolio? 

To  begin  to  answer  this  question,  we  drew  on  recent  insights  from  the  economics  research 
on  R&D  portfolio  choice  and  learning  value  (Guo,  2012).  A  standard  portfolio  choice  prob¬ 
lem  consists  of  investors  allocating  funds  across  traditional  financial  assets,  such  as  stocks  and 
bonds,  with  uncertain  rates  of  return.  In  a  dynamic  environment,  in  which  investments  are 
made  over  multiple  periods  of  time,  investors  learn  about  the  rate  of  return  of  various  assets 
and  rebalance  their  portfolios  to  incorporate  this  new  knowledge.  One  way  to  conceptualize 
this  learning  process  is  to  compare  investing  in  a  new  company  with  investing  in  an  established 
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Figure  5.1 

Trade-Offs  Between  Benefit-Cost  Index  and  Cultural  Acceptability 


Lowest  _  Highest 

ranked  ranked 

Benefit-cost  index 


RAND  flfl559*5. 1 


company.  An  investor  buying  stock  in  a  new  company  faces  a  high  degree  of  risk  (i.e.,  high 
uncertainty  about  the  rate  of  return).  Over  time,  as  the  company  becomes  more  established, 
investors  are  able  to  observe  multiple  rounds  of  performance  and  earnings,  diminishing  the 
level  of  uncertainty.  This  is  the  value  of  learning  about  the  return  of  an  uncertain  asset. 

Investment  in  research  differs  fundamentally  from  investment  in  stocks,  bonds,  and  other 
traditional  assets.  This  is  because,  with  traditional  assets,  the  predominant  assumption  is  that 
learning  occurs  passively  over  time  at  a  given  rate.  One  learns  by  reading  the  newspaper  and 
observing  the  market.  In  other  words,  the  rate  of  learning  is  decoupled  from  the  investment 
decision.  Investment  in  research  is  especially  unique  in  that  learning  about  the  returns  occurs 
actively  (i.e.,  the  more  you  invest  in  research,  the  more  you  learn).  At  an  extreme,  if  no  invest¬ 
ment  is  made,  one  is  never  able  to  learn  the  rate  of  return  from  a  research  investment.  The 
inevitable  conclusion  from  this  reasoning  is  that,  in  the  case  of  funding  research  investments, 
which  differs  from  funding  traditional  financial  investments,  the  value  of  learning  should  be 
incorporated  into  the  allocation  decision  process.  Thus,  although  a  primary  return  from  invest¬ 
ing  in  suicide  prevention  research  can  be  measured  using  a  benefit-cost  index,  a  secondary 
return  on  investment  is  the  value  of  learning,  which  should  also  be  taken  into  account  and  will 
vary  across  research  goals. 

Because  there  are  no  historical  data  on  the  uncertainty  of  returns  in  suicide  research,  we 
relied  on  expert  elicitation  from  the  internal  RAND  panel  (described  in  Chapter  Three)  to 
characterize  learning  value.  For  each  strategy,  the  panel  provided  guidance  on  assigning  three 
possible  learning  values  to  each  research  strategy:  high  learning  value  (greatest  uncertainty), 
medium  learning  value,  and  low  learning  value  (least  uncertainty).  Figure  5.2  shows  how 


56  Developing  a  Research  Strategy  for  Suicide  Prevention  in  the  Department  of  Defense 


Figure  5.2 

Benefit-Cost  Index  Relative  to  Future  Learning  Potential 
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learning  value  interacts  with  the  primary  metric  for  rate  of  return,  the  henefit-cost  index.  The 
11  research  strategies  are  ranked  in  terms  of  their  rate  of  return  (henefit-cost  index)  along  the 
vertical  axis,  and  the  length  of  individual  error  bars  reflects  the  learning  value  according  to  the 
return  on  investment. 

This  illustrative  graph  is  meant  to  convey  an  important  lesson:  Uncertainty  about  the 
returns  on  investment  precludes  us  from  establishing  an  exact  (“true”)  order  with  respect  to 
learning  potential.^  Although  we  may  be  confident  that  highly  ranked  strategies  generally  have 
greater  returns  than  lower  ranked  strategies,  we  are  less  confident  in  the  order  of  immediately 
adjacent  rankings.  In  fact,  learning  value  is  especially  important  when  choosing  between  two 
investments  that  are  closely  ranked.  Rankings  may  change  with  continued  investment  as  more 
information  is  collected.  For  example,  consider  two  research  strategies,  both  of  which  are  ini¬ 
tially  expected  to  have  the  same  rate  of  return  (suicides  prevented  per  cost).  Strategy  A  has 
higher  learning  value  (greater  uncertainty  represented  with  a  long  error  bar),  and  strategy  B  has 
less  learning  value.  Investment  in  each  strategy  results  in  further  learning  about  the  return  on 
investment,  which  reduces  the  uncertainty  associated  with  each  strategy. 

We  depict  this  dynamic  in  Figure  5.3.  The  narrowing  difference  between  the  dashed  line 
and  the  solid  line  represents  the  resolving  of  uncertainty.  Two  strategies,  formerly  tied  in  the 
rankings,  now  show  differentiation,  with  strategy  A  now  having  a  higher  rate  of  return  than 
strategy  B.  We  could  imagine  that,  after  a  great  deal  of  investment  in  each,  the  distributions 
would  narrow  enough  for  the  true  rankings  to  be  identified.  In  the  process,  the  research  port- 


^  Figure  5.2  is  illustrative  in  the  sense  that  we  do  not  assign  specific  values  to  the  level  of  uncertainty;  we  merely  depict  the 
relative  differences  in  uncertainty  across  the  strategies. 
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Figure  5.3 

Relationship  Between  Further  Investment  and  Degree  of  Uncertainty  in 
Two  Research  Strategies 


folio  manager  would  be  able  to  rebalance  the  portfolio  to  take  advantage  of  new  information 
about  the  rankings. 

Returning  to  Figure  5.2,  we  are  now  confronted  with  12  strategies  with  various  perceived 
learning  values.  Although  provider  training  ranked  first  in  terms  of  suicides  prevented  per  cost, 
research  in  this  field  is  perceived  to  be  mature  and  thus  has  a  small  learning  value.  Help-seeking 
ranked  slightly  lower  but  has  a  greater  learning  value.  Because  it  has  a  greater  learning  value 
(i.e.,  greater  uncertainty),  it  is  possible  the  maximum  potential  return  for  help-seekingis  greater 
than  the  maximum  potential  return  for  provider  training.  In  our  illustration,  this  is  represented 
by  the  top  of  the  error  bar  for  help-seeking,  which  is  higher  than  the  top  of  the  error  bar  for 
provider  training  Gatekeeper  training,  which  also  has  a  high  level  of  learning  value,  similarly 
offers  a  high  maximum  potential  return.  Population  risk-reduction  is  another  strategy  with  a 
high  learning  value,  and  its  error  bar  overlaps  those  of  several  adjacent  strategies  with  lower 
levels  of  learning  value. 

In  the  end,  an  optimal  portfolio  allocation  would  dedicate  the  most  resources  not  only  to 
the  strategies  with  the  highest  number  of  suicides  prevented  per  cost  but  also  to  the  strategies 
with  the  highest  learning  values.  Finally,  it  is  important  to  remember  that,  over  time,  learn¬ 
ing  from  investments  decreases  uncertainty  about  the  returns,  effectively  decreasing  the  length 
of  the  error  bars.  At  a  certain  point,  we  can  imagine  that  the  error  bars  will  shrink  until  they 
no  longer  overlap.  Eventually,  a  small  subset  of  research  strategies  will  emerge  as  “winners”  in 
terms  of  our  metric,  but  this  will  occur  only  after  research  investors  have  sunk  funds  into  each 
strategy. 
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Funding  and  Study  Allocation  Relative  to  the  Benefit-Cost  Index 

Similar  to  our  analyses  across  the  domains  of  importance,  effectiveness,  cultural  acceptability, 
cost,  and  future  learning  potential  in  Chapter  Four,  we  plotted  how  the  current  number  of 
research  studies  (Figure  5.4)  and  funding  levels  (Figure  5.5)  inside  and  outside  DoD  relate  to 
the  benefit-cost  index  we  constructed  for  each  aspirational  goal.  Here,  we  see  that  the  top  five 
ranked  by  our  index — provider  training,  help-seeking,  reduced  access  to  lethal  means,  gatekeeper 


Figure  5.4 

Number  of  Studies,  by  Benefit-Cost  Index 
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Figure  5.5 

Amount  of  Funding,  by  Benefit-Cost  Index 
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training,  and  preventing  reattempts — each  have  the  fewest  number  of  studies  and  lowest  level 
of  funding. 

Sensitivity  Analysis 

We  review  our  sensitivity  analysis  here,  hut  Appendix  E  provides  a  more  detailed  discussion. 
The  index  value  calculated  relies  on  parameters,  and  each  may  have  error  associated  with 
it.  Based  on  the  formula,  it  is  straightforward  to  infer  how  a  percentage  error  or  percentage 
change  in  the  parameters  will  affect  the  index.  Any  change  in  the  numerator  will  induce  an 
equal  change  in  the  index.  For  example,  a  5 -percent  individual  error  in  any  of  the  four  variables 
{n,  p,  s,  or  t)  or  a  5 -percent  combined  error  in  the  variables  in  the  numerator  will  translate  to 
a  5-percent  error  in  the  index.  Because  cost  is  in  the  denominator,  a  percentage  error  in  the 
denominator  requires  taking  the  inverse. 

We  estimated  the  relative  margin  of  error  associated  with  each  index  value  and  the  extent 
to  which  this  error  would  change  a  goal’s  ordinal  ranking  relative  to  the  goal  ranked  below  it. 
These  percentage  leads  are  presented  in  Table  5.7.  For  example,  the  difference  between  provider 
training  and  help-seeking  is  47  percent.  Thus,  our  estimates  for  both  goals  could  be  inaccurate, 
but  the  provider  training  index  would  have  to  be  47  percent  too  high,  the  help-seeking  index 
would  have  to  be  47  percent  too  low,  or  both  would  have  to  vary  to  a  proportional  degree  for 
the  ordinal  rankings  to  change.  For  certain  goals  {reduced  access,  psychosocial  interventions,  risk 
reduction,  and  quality  card),  the  relative  margins  are  close,  so  a  small  change  (<  15  percent) 
would  affect  the  rankings.  For  example,  if  the  risk  reduction  index  decreased  by  more  than 
1  percent,  it  would  fall  behind  quality  care  in  the  rankings. 

This  analysis  suggests  that  we  would  generally  expect  the  highest-  and  lowest-ranked 
research  goals  to  maintain  their  rankings,  given  reasonable  errors  in  the  assumptions  underly¬ 
ing  their  indexes.  However,  several  of  the  midranked  goals  have  indexes  that  are  relatively  close 


Table  5.7 

Percentage  Change  in  the  Benefit-Cost  Index 


Aspirational  Goal 

Benefit-Cost  Index 

%  "Lead"  Over 
Next  Lower 

Provider  training 

93.333 

47 

Help-seeking 

49.558 

96 

Reduced  access 

1.800 

7 

Gatekeeper  training 

1.667 

33 

Prevent  reattempts 

1.125 

35 

Psychosocial  interventions 

0.733 

8 

Risk  reduction 

0.672 

1 

Quality  care 

0.667 

10 

Continuity  of  care 

0.600 

33 

Screening 

0.400 

67 

Biological  interventions 

0.133 

63 

Prediction 

0.050 

— 
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together,  and  we  could  see  reordered  rankings  in  the  middle  given  a  different  set  of  parameter 
assumptions. 


Conclusion 

In  this  chapter,  we  integrated  findings  from  both  expert  panels  and,  applying  certain  assump¬ 
tions,  created  an  index  that  takes  into  account  not  only  the  explicit  henefit  (i.e.,  the  number  of 
suicides  prevented)  but  also  the  explicit  cost  of  implementing  an  intervention.  By  doing  so,  we 
were  able  to  rank  each  aspirational  research  goal  with  respect  to  a  metric  of  suicides  prevented. 
Furthermore,  we  paired  each  goal’s  beneht-cost  index  with  its  cultural  acceptability,  which  will 
inevitably  discount  certain  strategies  that  may  appear  effective  but  that  are  not  likely  to  gain 
traction  in  a  military  environment.  Certain  strategies,  such  as  help-seeking,  reduced  access,  and 
psychosocial  interventions,  rank  well  but  have  low  cultural  acceptability.  A  policymaker  may 
either  discount  such  strategies  or  work  on  making  them  more  culturally  acceptable.  Finally, 
we  refined  the  concept  of  “benefit”  by  characterizing  the  implicit  learning  value  of  pursuing  a 
research  program,  augmenting  the  explicit  benefits.  Gatekeeper  training  and  risk-reduction  are 
two  midranked  strategies  that  may  have  high  maximum  potentials  due  to  their  learning  value. 

Determining  the  proper  investment  model  is  especially  complicated  because  of  the  inher¬ 
ent  lack  of  traditional  data  sets  to  support  the  empirical  analysis  of  research  outcomes.  We  rec¬ 
ognize  that  there  are  many  unknowns;  our  methodology,  combined  with  expert  elicitation,  is 
designed  to  organize  the  unknowns  into  a  more  manageable  problem.  Our  primary  goal  was  to 
provide  and  populate  a  metric  in  a  held  lacking  such  a  metric  and  to  do  so  transparently.  While 
the  exact  composition  of  the  performance  index  and  the  assumptions  made  to  parameterize  the 
model  may  be  debated,  we  welcome  this  criticism.  Others  can  modify  the  model  or  change 
the  parameterization  with  their  own  assumptions  to  see  how  the  results  change.  We  believe 
that  this  process,  combined  with  our  expert  elicitation,  provides  a  solid  ground  on  which  to 
make  strategic  recommendations  to  inform  the  development  of  a  comprehensive  research  pro¬ 
gram  for  suicide  prevention  in  DoD. 

A  takeaway  is  that  investment  in  research  is  not  a  one-time,  static  process:  It  will  be 
repeated  over  many  periods,  and,  in  each  period,  more  results  will  be  observed  and  beliefs 
about  the  return  on  investment  will  be  updated.  The  recurring  process  of  investment,  learning 
from  the  investment,  and  updating  the  investment  strategy  in  fact  deemphasizes  the  impor¬ 
tance  of  coming  to  a  consensus  about  initial  parameters  and  emphasizes  the  importance  of 
adapting  investment  allocations  according  to  lessons  learned  along  the  way.  Only  over  suc¬ 
cessive  periods  of  investment  will  we  ultimately  identify  the  research  strategies  that  are  highly 
productive  and  those  that  will  are  dead  ends. 


CHAPTER  SIX 


Translating  Research  into  Practice 


The  ultimate  objective  of  any  suicide  prevention  research  study  is  to  provide  empirically  sup¬ 
ported  information  that  can  be  used  to  improve  or  inform  suicide  prevention  strategies.  Unfor¬ 
tunately,  there  is  a  research-practice  gap  in  disseminating  research  conducted  in  academic 
settings  to  individuals  who  may  actually  benefit  from  learning  about  the  research  (Lamb, 
Greenlick,  and  McCarty,  1998).  When  researchers  finish  collecting  data  for  a  research  study, 
they  typically  write  up  the  results  and  submit  an  article  for  publication  in  a  peer-reviewed  jour¬ 
nal.  Researchers  also  attend  local,  national,  or  global  conferences,  where  they  speak  with  others 
in  their  field  about  their  research  findings  and  learn  about  others’  research  findings.  Dissemi¬ 
nation  vehicles  include  formal  talks  or  presentations,  training  sessions,  poster  presentations,  or 
informal  talks  during  networking  activities.  However,  because  academic  conferences  and  peer- 
reviewed  journals  are  the  primary  tools  used  to  disseminate  research  findings  to  others,  the 
diffusion  of  this  research  into  actual  practice  can  be  hindered  for  several  reasons.  First,  many 
articles  in  journals  are  not  freely  available  to  the  public.  Second,  conferences  are  often  adver¬ 
tised  only  to  academics  or  those  who  subscribe  to  specific  mailing  lists;  they  are  also  expensive 
and  typically  require  travel.  Finally,  those  most  likely  to  use  the  findings  in  actual  practice 
(i.e.,  clinicians,  line  leaders,  and  providers  in  the  field)  typically  do  not  attend  academic  con¬ 
ferences  and  do  not  generally  read  journals,  partly  because  of  perceptions  that  such  research 
is  esoteric  and  not  easily  integrated  into  practice  (Carlton,  2012;  Sobell,  1996).  For  example, 
researchers  may  share  results  about  a  promising  suicide  intervention  in  the  literature  or  at  con¬ 
ferences,  but  line  leaders  and  providers  who  work  closely  with  service  members  and  may  benefit 
from  learning  about  the  intervention  do  not  receive  the  information  presented  at  the  confer¬ 
ences  or  do  not  know  where  to  find  published  research. 

Organizations  must  go  beyond  current  dissemination  practices  and  develop  theoretically 
and  empirically  sound  methods  for  diffusing  research  into  practice.  This  chapter  provides  an 
overview  of  evidence-informed  strategies  for  disseminating  research  into  practice,  with  the  goal 
of  helping  DoD  adopt  a  process  to  promote  such  diffusion  in  the  area  of  suicide  prevention.  We 
first  describe  the  methods  we  used  to  evaluate  how  successful  dissemination  occurs.  From  this 
process,  we  identified  ten  recommendations  to  effectively  diffuse  research  into  practice.  We 
describe  the  components  of  effective  diffusion,  review  how  current  practices  may  be  problem¬ 
atic  or  ineffective,  and  suggest  how  these  ten  components  may  help  overcome  current  barriers. 
Finally,  for  each  of  the  ten  components,  we  present  a  case  study  to  demonstrate  what  effective 
translation  of  research  looks  like  in  a  real-world  setting.  While  we  highlight  those  that  may 
be  specific  to  suicide  prevention,  we  took  a  more  general  approach  in  identifying  these  cases, 
drawing  from  several  disciplines. 
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Literature  Review  Strategy 

We  began  by  looking  through  the  research  literature  in  areas  including  but  not  limited  to  sui¬ 
cide  prevention  to  identify  successful  strategies  that  may  be  relevant  to  suicide  prevention  in 
DoD.  We  looked  at  research  across  areas  of  behavioral  health,  medicine,  criminal  justice,  edu¬ 
cation,  occupational  safety,  and  sociology  to  help  identify  strategies  to  facilitate  the  diffusion  of 
research  into  practice.  We  conducted  a  systematic  literature  search  using  the  MEDLINE  and 
EBSCOhost  databases,  as  well  as  Google  Scholar,  for  studies  related  to  “translating  research 
into  practice”  or  “diffusion  of  research.”  We  selected  articles  and  reports  published  within  the 
prior  ten  years  (2002-2012)  but  also  included  classic  work  identified  from  the  reference  sec¬ 
tions  of  these  articles.  The  search  yielded  approximately  80  unique  articles  and  reports  that 
were  relevant  to  the  targeted  topic. 

Although  we  used  this  literature  search  to  identify  case  studies,  we  augmented  this  infor¬ 
mation  by  holding  discussions  with  experts  at  RAND  in  the  areas  of  education,  substance 
abuse,  occupational  safety,  criminal  justice,  behavioral  health,  and  health  care  to  locate  case 
studies  of  research  programs  that  have  been  successfully  implemented  into  practice.  We  sum¬ 
marize  these  case  studies  later  in  this  chapter  and  highlight  the  components  of  effective  dif¬ 
fusion  supported  in  the  theoretical  and  empirical  literature  on  putting  research  into  practice. 

A  Note  on  Terminology 

Although  we  searched  the  literature  across  disciplines,  much  of  the  relevant  research  focuses 
on  health  care  topics.  As  such,  we  use  the  term  providers  loosely;  providers  may  include,  for 
example,  installation  leaders  who  want  to  implement  new  suicide  prevention  trainings  in  their 
commands.  We  also  refer  often  to  adopters  or  adopters  on  the  ground — descriptions  that  encom¬ 
pass  leaders,  providers,  and  other  individuals  who  are  actually  involved  in  implementing  a 
research  program  on  an  installation  or  within  an  organization  at  a  particular  site.  Finally,  we 
use  the  terms  program,  intervention,  and  innovation  interchangeably  to  highlight  how  these 
components  are  applicable  to  an  array  of  practices. 


The  Ten  Components 

The  research  literature  supports  ten  components  that  organizations  and  institutions  can  use  to 
effectively  diffuse  research  into  practice  (see  Table  6.1).  We  elected  to  not  use  the  phrase  neces¬ 
sary  components,  because  not  all  ten  components  may  be  required  in  all  instances.  That  is,  not 
every  research  study  or  collection  of  studies  will  require  all  ten,  and  it  will  be  important  to 
consider  why  or  why  not  each  of  these  ten  components  is  addressed  when  attempting  to  dif¬ 
fuse  research  findings  into  practice.  The  components  are  presented  in  a  proposed  sequential 
order  and  not  in  order  of  importance.  These  components  will  likely  not  follow  a  specific  order 
in  real-world  implementation;  thus,  they  should  be  understood  as  a  guide  and  not  a  strict 
chronological  structure.  They  can  be  thought  of  as  encompassing  three  broad  areas,  with  com¬ 
ponents  that  (1)  demonstrate  that  the  program  is  needed,  could  be  successful  at  a  particular 
site,  and  is  rooted  in  scientific  evidence;  (2)  relate  to  infrastructure,  such  as  internal  leadership 
and  financial  support  for  the  program;  and  (3)  relate  to  garnering  support  and  dissemination 
at  the  ground  level. 
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Table  6.1 

Ten  Components  Recommended  to  Effectively  Diffuse  Research  into  Practice 


Category 

Component 

Program  is  needed,  could  be 

1. 

Evidence  of  an  identified  need 

successful,  and  is  rooted  in 
scientific  evidence 

2. 

Evidence  of  research  quality 

3. 

Evidence  of  real-world  effectiveness 

Supportive  infrastructure 

4. 

Leadership  buy-in  and  support  from  key 
stakeholders 

5. 

Funding  or  other  institutional  support 

Ground-level  support  and 

6. 

Collaboration  with  credible  sponsors 

dissemination 

7. 

Provision  of  incentives  or  development  of  policies 

8. 

Peer  network  supportive  of  adoption 

9. 

Dissemination  materials 

10. 

Expectation  of  a  cultural  shift 

We  describe  each  of  the  components  in  turn.  For  each,  we  first  provide  the  theoretical  and 
empirical  support  for  the  strategy,  followed  by  one  or  two  case  studies  that  exemplify  how  the 
specific  component  was  used  (along  with  others)  to  effectively  integrate  research  findings  into 
routine  practice.  Some  of  the  methodologies  and  designs  of  these  case  studies  may  apply  spe¬ 
cifically  to  the  military’s  strategy;  however,  the  case  studies  are  intended  to  describe  the  process 
of  diffusion  and  demonstrate  the  successful  translation  of  research  into  practice. 

1.  Evidence  of  an  Identified  Need 

The  first  step  in  effectively  translating  research  into  practice  involves  establishing  a  need  for  an 
innovation.  One  of  the  primary  models  guiding  the  diffusion  of  innovation  suggests  five  key 
factors  that  increase  the  likelihood  that  an  innovation  will  be  adopted  (Rogers,  2003).  The  first 
is  relative  advantage,  which  refers  to  perceptions  that  the  innovation  meets  a  particular  need  or 
will  be  significantly  better  than  current  practice.  In  other  words,  a  program  seeking  adoption 
must  be  perceived  as  novel  and  necessary  to  improve  upon  current  practice. 

Individuals  and  systems  adopting  the  program  may  need  to  understand  that  what  is  cur¬ 
rently  being  practiced  may  be  ineffective,  unhelpful,  or  even  harmful.  There  may  be  general 
consensus  that  the  current  practice  is  not  working,  but  it  may  also  be  that  individuals  need 
to  first  be  convinced  that  a  need  exists.  One  important  way  to  demonstrate  a  need  for  a  new 
program  is  to  collect  and  present  data  to  senior  leaders,  decisionmakers,  and  those  expected  to 
adopt  the  program  to  demonstrate  evidence  that  the  current  practice  is  not  helpful,  is  lacking, 
or  could  be  improved.  This  process  can  begin  the  discussion  that  a  change  needs  to  be  made 
as  gaps  in  the  current  practice  are  highlighted  and  the  needs  of  the  organization  are  assessed 
(Hempel  et  ak,  2012).  Because  providers  and  line  leaders  may  be  frustrated  because  current 
practices  are  ineffective  in  helping  individuals  with  a  particular  problem,  demonstrating  the 
relative  advantage  of  a  new  program  may  also  encourage  adoption  by  those  on  the  ground. 

After  implementing  the  new  program,  collecting  data  on  its  impact  can  help  provide 
empirical  evidence  of  effectiveness,  increase  credibility,  and  retain  organizational  support. 
Data  from  an  adopted  program,  as  opposed  to  data  on  programs  not  already  adopted,  may 
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be  more  convincing  for  senior  leaders,  because  they  can  see  how  well  the  program  they  have 
invested  in  is  working  (Barth,  2004;  Bradley  et  al.,  2004).  Continuous  evaluation  of  the  pro¬ 
gram  is  necessary  to  ensure  that  senior  leaders  and  providers  remain  aware  of  the  effectiveness 
of  the  program  after  the  trial  period.  Boxes  6.1  and  6.2  present  two  case  studies  that  illustrate 
this  first  component:  employee  assistance  programs  (EAPs)  and  the  Fleetwood  Project. 

2.  Evidence  of  Research  Quality 

A  plan  for  effective  diffusion  begins  with  an  understanding  of  how  to  identify  quality  research. 
Prior  to  implementing  a  program,  it  is  essential  to  understand  how  the  program  or  intervention 

Box  6.1 

Case  Study:  Employee  Assistance  Programs 


Identified  Need 

Employers  were  increasingly  recognizing  that  alcohol-use  problems  were  affecting 
employees'  work  performance  (Roman  and  Blum,  2002).  It  was  believed  that  alcohol- 
use  problems  were  leading  to  reduced  work  productivity  through  missed  work  days 
and  inefficient  output. 

Innovation 

EAPs  are  employer-sponsored  programs  that  provide  assessment,  counseling  support, 
and  referrals  for  additional  resources  to  employees  and  members  of  their  households. 
While  EAPs  began  in  the  1970s  to  help  identify  and  reduce  substance  abuse  among 
employees,  they  have  expanded  to  provide  workers  with  services  for  mental  health 
concerns,  stress,  health  care  concerns,  financial  and  legal  concerns,  work  relationship 
issues,  and  family  relationship  issues.  EAPs  are  typically  free  to  employees,  are  included 
in  standard  benefits  packages,  and  refer  individuals  to  outside  providers  who  can  assist 
them  in  a  limited  number  of  sessions. 

Research 

Support 

EAPs  reduce  work  loss  in  60  percent  of  cases,  with  an  average  gain  in  work  productivity 
of  43  percent  (Attridge,  2001,  2002). 

Key  Materials 

The  Center  for  Prevention  and  Health  Services  published  An  Employer's  Guide  to 

Employee  Assistance  Programs  to  help  employers  understand  the  value  and  cost- 
effectiveness  of  providing  their  employees  with  EAPs.  The  publication  presents  research 
findings  on  EAPs  to  help  employers  make  informed  decisions  about  implementing 
these  programs.  The  center's  EAP  Workgroup  helped  employers  understand  that  EAPs 
represent  a  first-line  response  to  providing  prevention,  triage,  and  short-term  problem- 
resolution  services.  Surveys  demonstrate  the  cost-effectiveness  of  these  programs, 
highlighting  that  the  initial  costs  associated  with  establishing  EAPs  outweigh  the  long¬ 
term  costs  associated  with  the  problems  that  they  prevent  (Marsh  USA  and  Mercer 

Health  and  Benefits,  2008;  Watson  Wyatt  Worldwide,  2007). 

Status  of 

Diffusion 

An  estimated  76  percent  of  large  companies  representing  more  than  14  million 
employees  provide  an  EAP  as  part  of  the  standard  benefit  package  (Hartwell  et 
al.,  1996).  The  majority  (81  percent)  of  EAPs  are  external  (i.e.,  located  outside  the 
workplace),  and  about  17  percent  of  companies  have  an  internal  EAP.  The  majority  of 
employers  with  EAPs  see  their  utility  (Rothermel  et  al.,  2008). 

Key  Components 
of  Diffusion 

Evidence  of  an  identified  need:  Poor  work  productivity  because  of  substance  abuse 
by  workers,  need  for  substance-abuse  treatment  for  employees,  and  documented 
improvements  in  work  productivity 

Evidence  of  effectiveness:  Recognized  benefits  of  targeting  substance-use  problems  to 
reduce  the  costs  associated  with  poor  job  performance 

Dissemination  materiais:  Educated  employers  about  EAPs  through  training,  published 
guide  for  employers  with  accessible  research  findings,  convened  EAP  Workgroup 

Translating  Research  into  Practice  65 


Box  6.2 

Case  Study:  The  Fleetwood  Project 


Identified  Need 

There  was  a  need  to  reduce  inappropriate  medication  use,  undertreatment  of  common 
diseases,  and  adverse  drug  events  among  patients  in  nursing  homes  (Harms  and 

Garrand,  1998).  External  pharmacists  and  prescribing  clinicians  at  the  facilities  typically 
did  not  inform  each  other  of  patients'  treatment  or  drug  regimen,  which  could  lead  to 
complications. 

Innovation 

The  Fleetwood  Project  is  a  model  of  long-term  pharmacy  care  used  in  nursing  homes  in 
Wisconsin  and  North  Carolina  (Daschner  et  al.,  2000;  Lapane  et  al.,  2011).  It  identifies 
nursing  home  residents  at  the  highest  risk  for  preventable  adverse  drug  events  and 
combines  these  assessments  with  formalized  pharmaceutical  care  planning  (e.g., 
establishing  a  drug  regimen)  and  direct  communication  between  the  consultant 
pharmacist  and  the  prescribing  clinician.  It  recognizes  that  pharmacists  have  specialized 
training  and  may  be  able  to  contribute  to  a  patient's  care,  rather  than  simply  filling 
a  prescription  as  ordered  by  a  clinician.  Dispensing  pharmacists  and  consulting 
pharmacists  who  typically  did  not  communicate  with  each  other  were  required  to 
collaborate. 

Research 

Support 

The  Fleetwood  Project  has  helped  increase  the  quality  of  drug-related  care,  decrease 
mortality  rates,  and  significantly  save  costs.  More  specifically,  the  program  has 
increased  the  number  of  patients  who  experience  optimal  therapeutic  outcomes  by 

43  percent  and  saved  $3.6  billion  annually  in  costs  from  avoided  medication-related 
problems  (Bootman,  Harrison,  and  Cox,  1997). 

Key  Materials 

The  program  has  a  toolkit  available  online  and  serves  as  a  model  for  other  nursing 
homes.  The  program  has  also  developed  the  Web-Based  Pharmaceutical  Care  Software 
to  facilitate  the  exchange  of  information  between  internal  and  external  pharmacists. 

Status  of 

Diffusion 

The  program  is  being  evaluated  for  effectiveness  in  26  nursing  facilities  in  North 

Carolina.  The  Commonwealth  Fund  and  the  Retirement  Research  Foundation  funded 
the  project,  with  grants  totaling  more  than  $750,000. 

Key  Components 
of  Diffusion 

Evidence  of  an  identified  need:  Identified  a  clear  purpose  for  implementing  a  drug 
review  process  and  improving  care,  which  fulfilled  an  established  need  and  helped 
attract  substantial  financial  and  administrative  resources,  including  a  staff  specifically 
to  disseminate  the  program 

Evidence  of  research  quality:  Model  tested  in  Wisconsin  and  North  Carolina  nursing 
homes 

Leadership  buy-in  and  support  from  key  stakeholders:  Strong  senior  leadership 
championing  the  approach  and  senior  leaders  at  each  facility  helping  to  reengineer  the 
nursing  home  pharmacies 

Funding  or  other  institutional  support:  Dedicated  funding  for  program  staff 

Peer  network  supportive  of  adoption;  Presence  of  clinical  champion  to  support  staff, 
reduce  resistance  from  peers,  and  work  with  other  clinicians  to  promote  buy-in; 
provided  trainings 

Collaboration  with  credible  sponsors:  Collaborative  approach  across  disciplines  as 
pharmacists  and  nursing  home  staff  were  trained  to  enhance  communication  skills  and 
increase  their  knowledge  of  technologically  advanced  approaches  to  share  data  among 
providers 

Innovative  and  creative  approaches:  Used  technology  to  expedite  communication 
between  pharmacists  and  nursing  home  staff 

Expectation  of  a  cultural  shift:  Changed  status  quo  but  helped  target  an  established 
need 
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is  supported  by  research.  There  are  both  nonprofit  and  government-supported  entities  whose 
responsibilities  include  reviewing  the  research  evidence  for  different  programs  and  practices, 
some  of  which  exclusively  focus  on  suicide  prevention  and  some  of  which  focus  on  suicide  pre¬ 
vention  in  addition  to  other  areas.  Here,  we  profile  a  sampling  of  these  organizations. 

SAMHSA's  National  Registry  of  Evidence-Based  Programs  and  Practices 

SAMHSA’s  online  National  Registry  of  Evidence-Based  Programs  and  Practices  offers  descrip¬ 
tions  of  291  mental  health  and  substance  abuse  interventions  and  the  resources  that  support 
them  (see  SAMHSA,  2014).  The  quality  of  the  research  evidence  supporting  each  intervention 
is  rated  by  independent  reviewers  for  the  reliability  of  the  measures,  the  validity  of  the  mea¬ 
sures,  intervention  fidelity,  missing  data  and  attrition,  potential  confounding  variables,  and  the 
appropriateness  of  the  analysis.  Independent  reviewers  also  assess  the  intervention’s  readiness 
for  dissemination  based  on  the  availability  of  implementation  materials,  training  and  support 
resources,  and  quality  assurance  procedures.  Costs  of  the  program  and  contact  information  are 
also  provided.  The  registry  can  be  accessed  at  http://www.nrepp.samhsa.gov. 

The  Suicide  Prevention  Resource  Center's  Best  Practices  Registry 

The  Suicide  Prevention  Resource  Center’s  Best  Practices  Registry  is  based  on  National  Regis¬ 
try  of  Evidence-Based  Programs  and  Practices  suicide-focused  interventions.  It  provides  a  list 
of  evidence-based  programs,  along  with  guidelines  and  protocols  based  on  expert/consensus 
statements  from  the  field.  The  registry  can  be  accessed  at  http://www.sprc.org/bpr. 

The  Cochrane  Collaboration 

The  Cochrane  Collaboration  offers  reviews  of  empirically  supported  interventions  and  prac¬ 
tices  in  the  health  care  field.  The  collaboration’s  online  library  offers  research  summaries, 
full-text  reviews,  journal  articles  on  controlled  trials,  and  podcasts  (see  Cochrane  Collabo¬ 
ration,  2012).  Its  reviews  are  specifically  targeted  toward  policymakers,  practitioners,  and 
consumers  and  are  intended  to  provide  objective  evidence  on  treatments  in  a  particular  area 
of  health  care.  The  collection,  along  with  user  guides  for  practitioners,  can  be  accessed  at 
http://www.cochrane.org/information-practitioners. 

U.S.  Preventive  Services  Task  Force 

The  AHRQ-supported  U.S.  Preventive  Services  Task  force  is  a  group  of  nationally  recognized 
experts  in  the  field  of  clinical  preventive  services.  The  task  force  makes  recommendations 
about  preventive  services  based  on  the  research  evidence  on  such  topics  as  health  care  screen¬ 
ing,  counseling  services,  and  preventive  medications.  The  group  has  created  fact  sheets,  videos, 
slide  presentations,  and  reports  to  assist  providers  and  clinicians  in  making  evidence-informed 
decisions  in  their  practices  (see  AHRQ,  2012).  An  overview  of  available  resources  can  be  found 
at  http://www.ahrq.gov/professionals/clinicians-providers/guidelines-recommendations/uspstf 

VA  Quality  Enhancement  Research  Initiative  (QUERI) 

The  VA  Quality  Enhancement  Research  Initiative  is  focused  on  using  research  evidence 
to  improve  clinical  practice  for  veterans  (see  U.S.  Department  of  Veterans  Affairs,  Quality 
Enhancement  Research  Initiative,  2014).  The  initiative  focuses  efforts  addressing  ten  high- 
risk  or  high-prevalence  conditions,  such  as  diabetes,  mental  health,  and  substance-use  disor¬ 
ders.  QUERI’s  online  Guide  for  Implementing  Evidence-Based  Clinical  Practice  and  Conducting 
Implementation  Research  outlines  methods  and  resources  for  putting  research  into  practice. 
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These  guidelines  are  tailored  specifically  toward  clinical  interventions  and  screenings  for  the 
ten  high-risk  or  high-prevalence  conditions  among  veterans.  For  more  information  about 
QUERI,  see  http://www.queri.research.va.gov. 

Assessing  Research  Quality 

There  are  a  number  of  factors  that  should  be  considered  when  assessing  the  quality  of  research. 
In  Appendix  F,  we  provide  a  brief  summary  of  the  types  of  research  domains,  research 
approaches,  and  research  findings.  However,  studies  that  use  randomized  controlled  trials 
(RCTs)  are  the  gold  standard  for  quality  research  (Lohr,  2004)  and  warrant  specialized  dis¬ 
cussion  here.  The  organizations  identified  in  this  section  use  these  standards  when  evaluating 
research  evidence  in  a  particular  area. 

RCTs  are  intervention  studies  in  which  one  group  typically  receives  an  intervention 
(e.g.,  counseling,  medication)  and  a  control  group  receives  something  besides  the  tested  inter¬ 
vention  (e.g.,  usual  care,  placebo  treatment,  assessments  only).  RCTs  that  are  of  particularly 
high  quality  should  clearly  answer  the  following  questions:* 

1.  Study  population.  Who  is  being  studied?  Why  is  it  important  to  study  this  particular 
group?  Is  the  sample  included  in  the  study  representative  of  the  targeted  population? 
Identifying  the  study  population  is  essential  to  understanding  how  the  findings  can  be 
generalized  to  populations  that  may  be  similar  to,  or  different  from,  the  study  sample. 

2.  Randomization.  Are  participants  assigned  randomly  to  experimental  and  control  con¬ 
ditions  and  not  based  on  any  other  factors?  Efforts  should  be  made  to  ensure  that  ran¬ 
domization  was  not  compromised  (e.g.,  that  participants  in  the  control  condition  did 
not  “cross  over”  to  the  intervention  group)  and  to  verify  that  there  were  no  differences 
in  demographics  or  important  outcome  data  between  the  groups  at  baseline. 

3.  Blinding.  Are  participants,  providers,  data  coders,  and  other  study  staff  blind  to  partici¬ 
pants’  assigned  condition?  Ensuring  that  the  individuals  involved  in  the  study  are  blind 
to  patients’  conditions  adds  an  extra  layer  of  control  to  reduce  confounding  influences 
or  expectations. 

4.  Replicability.  Is  the  intervention  described  in  detail  so  that  replication  studies  are  pos¬ 
sible?  Have  researchers  in  others  laboratories  been  able  to  find  similar  effects?  Good 
RCTs  are  able  to  demonstrate  replicability  across  different  samples  and  across  different 
research  teams. 

5.  Intervention  fidelity.  What  measures  did  the  researchers  take  to  ensure  that  the  interven¬ 
tion  was  delivered  consistently  across  providers?  The  use  of  manuals,  supervision,  and 
fidelity  checks  can  ensure  standardization  of  the  intervention  among  providers. 

6.  Reliability  and  validity  of  outcome  measures.  Did  the  researchers  utilize  measures  that 
have  established  reliability  and  validity  with  the  target  population  in  prior  work?  If 
established  measures  with  research  support  exist,  it  is  better  if  researchers  use  them  to 
facilitate  comparability  across  studies.  It  is  also  important  to  ensure  that  the  study  uses 
validated  and  “real-world”  measures  to  examine  outcomes.  When  possible,  self-reports 
should  be  corroborated  by  independent  and  objective  measures  (e.g.,  tests  of  blood  alco¬ 
hol  level  to  corroborate  self-reported  alcohol  use). 


*  This  list  is  adapted  from  Coalition  for  Evidence-Based  Policy,  2003,  and  West  et  ah,  2002. 
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7.  Appropriateness  of  statistical  analyses.  Was  the  analytic  plan  appropriate  and  accurate? 
Good  results  rely  on  up-to-date  scientific  analyses  that  provide  meaningful  statistics 
that  can  he  compared  across  studies.  Statistical  effects  should  he  reported  whenever  the 
research  suggests  a  difference  between  the  intervention  and  control  groups.  The  gener¬ 
ally  accepted  level  of  statistical  significance  is  0.05,  which  means  that  there  is  a  one  in 
20  prohahility  that  the  differences  observed  between  intervention  and  control  outcomes 
were  by  chance.  These  effects  should  be  reported  in  real-world  terms,  and  both  signifi¬ 
cant  and  nonsignificant  findings  related  to  a  priori  hypotheses  should  be  reported. 

8.  Follow-up  assessment  of  outcomes.  Did  the  researchers  examine  the  lasting  effects  of  the 
intervention?  How  long  were  the  follow-up  periods?  Long-term  outcomes  (e.g.,  two  to 
three  years  after  the  intervention)  are  preferred  over  immediate  outcomes  or  short-term 
outcomes  (e.g.,  one-month  post-intervention  assessment). 

9.  Attrition.  How  many  participants  were  lost  at  follow-up?  How  were  missing  data  han¬ 
dled  in  statistical  analyses?  The  percentage  of  participants  lost  over  time  should  be  low 
and  checked  to  determine  whether  dropout  was  the  result  of  the  intervention  (e.g., 
whether  participants  dropped  out  because  the  intervention  was  causing  too  much  dis¬ 
tress).  Outcomes  should  be  evaluated  even  for  those  who  do  not  complete  the  experi¬ 
mental  intervention,  when  available. 

10.  Limitations.  What  confounders  exist  that  may  also  explain  the  findings?  How  might 
these  factors  affect  the  results?  No  research  study  is  flawless,  and  researchers  should 
indicate  the  limitations  of  their  study  so  that  they  can  be  addressed  in  future  work. 

In  sum,  being  able  to  recognize  quality  research  is  essential  so  that  untested  (or  poten¬ 
tially  harmful)  programs  are  not  widely  disseminated.  There  are  several  nonprofit  and  govern¬ 
ment  supported  entities  that  review  the  research  for  programs  in  a  particular  area  of  interest 
and  make  recommendations  for  the  use  of  empirically  supported  programs  in  practice.  Given 
the  research-practice  gap,  it  is  essential  to  find  low-cost  or  free,  accessible  methods  to  dissemi¬ 
nate  findings  from  empirically  based  research  to  inform  those  on  the  ground  who  are  looking 
to  adopt  a  new  program. 

Box  6.3  presents  a  case  study  illustrating  this  second  component:  dialectical  behavior 
therapy. 

3.  Evidence  of  Effectiveness 

An  innovation  can  go  from  a  highly  controlled  laboratory  environment  that  does  not  mimic 
real-world  settings  (efficacy  research)  to  real-world  settings  for  individuals  in  the  general  popu¬ 
lation  (effectiveness  research).  Effcacy  research  typically  involves  small-scale  studies  in  which 
participants  have  to  meet  strict  inclusion  and  exclusion  criteria,  making  such  participants  gen¬ 
erally  quite  different  from  individuals  in  the  overall  community  of  interest.  The  expectation  is 
that  the  observed  effects  will  translate  to  the  general  population,  but  this  is  not  always  tested. 
In  effectiveness  research,  studies  come  out  of  the  laboratory  and  into  the  real  world  with  less 
stringent  criteria  for  participation.  It  is  effectiveness  studies  that  provide  the  richest  informa¬ 
tion  about  potential  treatment  effects  for  the  general  population.  Effectiveness  studies  help  test 
whether  research  programs  can  be  translated  successfully  to  real-world  settings  and  whether 
the  programs  have  the  components  necessary  to  achieve  such  diffusion.  Effectiveness  research 
is  akin  to  what  Rogers  (2003)  refers  to  as  “trialability,”  or  how  much  the  new  practice  can 
be  “trialed”  or  tested  out  before  the  final  decision  about  whether  to  standardize  the  practice. 
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Box  6.3 

Case  Study:  Dialectical  Behavior  Therapy 


Identified  Need 

Borderline  personality  disorder  (BPD)  is  a  psychiatric  disorder  marked  by  an  attempt  to 
control  strong  emotions  with  self-harm  behaviors  (e.g.,  cutting)  or  suicide  attempts. 

The  severity  and  number  of  symptoms  in  BPD  patients  makes  the  disorder  particularly 
difficult  to  treat  through  standard  cognitive  behavioral  therapy. 

Innovation 

Marsha  Linehan  at  the  University  of  Washington  in  Seattle  developed  dialectical 
behavior  therapy  (DBT),  an  empirically  based,  manual-guided  treatment  for  individuals 
with  BPD.  Linehan  began  modifying  cognitive  behavioral  strategies  in  a  structured 
way  to  treat  these  patients.  She  used  these  modified  cognitive  behavioral  strategies 
in  conjunction  with  mindfulness  techniques  with  her  most  difficult  patients  and  noted 
that  they  began  to  improve.  In  1993,  she  published  her  strategies  in  a  manual  intended 
as  a  guide  for  other  therapists  (Linehan,  1993).  This  initial  study  was  followed  by  a  series 
of  RCTs  to  empirically  examine  the  effects  of  DBT  on  BPD  patients. 

Research 

Support 

Between  1991  and  2011,  Linehan  and  her  colleagues  designed  and  implemented  more 
than  15  RCTs  by  recruiting  patients  who  were  seeking  therapy  for  BPD,  substance 
abuse,  trauma,  or  suicidal  behaviors  into  small  research  studies  that  tested  DBT  against 
a  treatment  as  usual  (e.g.,  cognitive  behavioral  therapy)  or  a  waitlist  control  condition 
in  which  participants  waited  for  a  period  of  time  before  receiving  care.  Effectiveness 
trials  emerged  later.  For  example,  DBT  was  tested  with  patients  hospitalized  for  suicide 
attempts  (Bohus  et  al.,  2004). 

Key  Materials 

To  promote  DBT  beyond  journal  readers,  Linehan  created  a  center  on  the  University 
of  Washington  campus  to  train  therapists  and  develop  research  on  DBT.  She  attended 
local  and  national  conferences  and  created  a  website  to  collect  DBT  research  and  clinical 
resources,  announce  training  opportunities  for  therapists,  and  provide  interested 
patients  with  therapy  options.  Multiple  manuals  for  providers  and  patient  workbooks 
have  since  been  published. 

Status  of 

Diffusion 

DBT  is  now  the  standard  treatment  for  patients  with  BPD  or  suicidal  behaviors  in 
the  United  States  and  has  gained  popularity  in  several  European  countries,  Canada, 
Australia,  New  Zealand,  and  China.  DBT  has  had  widespread  exposure  and  been 
implemented  outside  the  University  of  Washington  clinic  (e.g.,  in  hospitals,  crisis 
services,  forensic  units;  Swenson,  2000). 

Key  Components 
of  Diffusion 

Evidence  of  research  quality;  Published  RCTs  in  top-tier  journals,  disseminated  research 
findings  via  public  website,  conferences,  and  trainings;  strong  empirical  support  for  an 
area  with  no  prior  research 

Evidence  of  an  identified  need:  Fit  a  need  for  providers  who  were  "burned  out" 
by  difficult  patients,  appealed  to  providers  from  multiple  theoretical  backgrounds; 
standardized,  manualized,  and  made  accessible  to  providers  at  all  levels;  lack  of  other 
promising  empirical  treatments  for  BPD  patients 

Evidence  of  real-world  effectiveness:  Implementation  outside  the  research  environment 
in  communities;  reimbursement  from  managed-care  companies 

Leadership  buy-in  and  support  from  key  stakeholders:  Managed-care  companies 
reimburse  for  DBT  if  there  is  a  BPD  diagnosis;  support  from  multiple  institutions 

Funding  or  other  institutional  support:  Endorsement  of  the  therapy  by  several  state 
departments  of  mental  health 

Peer  network  supportive  of  adoption:  Supportive  community  of  DBT  therapists  to  assist 
with  difficult  patients 

Provision  of  incentives  or  development  of  policies;  Manualized  structure  to  treat  a 
disorder  that  many  clinicians  are  admittedly  inept  at  treating 

Dissemination  materials:  Publication  of  a  DBT  manual  and  workbook  for  patients, 
trainings  by  the  developers,  and  online  resources 

Expectations  of  a  cultural  shift:  Blends  many  different  types  of  theoretical  orientations, 
which  allows  providers  with  diverse  training  backgrounds  to  find  something  familiar  in 
the  practice 
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Having  the  ability  to  easily  pilot-test  a  new  practice  may  affect  the  degree  to  which  the  guide¬ 
line  is  followed  in  the  future. 

Unfortunately,  programs  that  are  complex  and  highly  standardized  for  the  research  set¬ 
ting  may  be  difficult  to  implement  in  real-world  settings  (Glasgow,  Lichtenstein,  and  Marcus, 
2003).  Many  efficacy  studies  are  not  tested  for  effectiveness  in  the  population  or  fail  to  show 
similar  effects  when  tested  for  effectiveness.  Also,  some  providers  or  those  expected  to  adopt 
the  new  program  may  hold  the  belief  that  research  studies  have  limited  real-world  generaliz- 
ability  and  may  express  uncertainty  or  disinterest  in  a  new  approach  (Pagoto  et  ah,  2007;  Riley 
et  ah,  2007;  Stewart,  Stirman,  and  Chambless,  2012).  Developing  trainings  to  help  these  pro¬ 
viders  understand  the  importance  of  effectiveness  research  may  assist  with  adoption. 

Box  6.4  presents  one  case  study  that  illustrates  this  third  component:  the  Provonost 
Checklist. 

4.  Leadership  Buy-In  and  Support  from  Key  Stakeholders 

Strong  support  at  the  senior  management  level  within  an  organization  adopting  the  research 
strategy  is  important  for  several  reasons:^ 

1.  Leaders  can  provide  financial  support  and  help  manage  changes  among  personnel  in  adopt¬ 
ing  evidence-based  practices.  Adoption  of  a  new  program  may  involve  deviating  from 
current  practices.  Leaders  can  facilitate  the  changes  necessary  at  a  particular  site 
(e.g.,  hiring  new  staff,  providing  training)  and  can  generate  or  advocate  for  hnancial 
support. 

2.  Leaders  can  model  engagement  with  research  practices.  Through  early  adoption  of  prac¬ 
tices  by  leadership,  later-stage  adopters  of  an  innovation  can  observe  the  changes  made 
within  an  organization  and  see  how  the  program  may  work. 

3.  Leaders  can  establish  incentives  for  attending  discussions  or  trainings  to  hear  about  novel 
research-based  approaches.  Adopters  on  the  ground  can  be  encouraged  and  incentivized 
(e.g.,  with  financial  incentives,  training  certificates,  freedom  from  duties  for  time  spent) 
to  learn  about  empirically  supported  approaches  implemented  both  inside  and  outside 
the  organization. 

4.  Leaders  can  develop  kickoff  meetings,  bulletins,  fact  sheets,  and  meetings  that  increase 
involvement  in  evidence-based  care.  Leadership  can  promote  awareness  of  the  new  prac¬ 
tices  through  dissemination  strategies  aimed  at  increasing  the  visibility  of  the  innova¬ 
tion  to  facilitate  speedy  adoption. 

5.  Leaders  can  spread  the  word  to  others  (i.e.,  within  their  organization  or  externally)  who 
are  interested  in  new  programs.  Leaders  can  hold  formal  and  informal  discussions  to 
promote  awareness  of  a  new  program,  both  with  adopters  on  the  ground  within  their 
organization  and  with  leaders  at  similar  organizations. 

6.  Leaders  can  create  a  network  of  supportive  individuals,  both  inside  and  outside  their  orga¬ 
nization,  with  similar  interests.  Leaders  and  adopters  on  the  ground  can  work  collabora- 
tively  to  support  each  other  during  the  implementation  of  a  new  program.  Leaders  at 
similar  organizations  who  have  adopted  the  program  can  share  what  has  worked  and 
what  has  not  worked  at  their  site. 


^  This  list  is  adapted  from  Bradley  et  ah,  2004;  Owen  et  ah,  2006;  Oxman  et  ah,  1995;  Rogers,  2003;  and  Yuan  et  ah, 
2010. 
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Box  6.4 

Case  Study:  The  Pronovost  Checklist 


Identified  Need 

Each  year,  central  venous  catheters  given  to  patients  in  intensive  care  units  (ICUs)  are 
estimated  to  cause  80,000  bloodstream  infections  and  up  to  28,000  deaths. 

Innovation 

An  anesthesiologist  at  Johns  Hopkins  University  named  Peter  Pronovost  developed 
a  five-step  checklist  to  minimize  infections  contracted  by  patients  in  hospitals  who 
received  a  central  venous  catheter.  The  simple  checklist  reminded  health  care  providers 
to  (1)  wash  their  hands  prior  to  performing  the  procedure,  (2)  clean  the  patient's  skin, 

(3)  put  sterile  drapes  over  the  patient,  (4)  wear  sterile  gloves  and  equipment,  and 
(5)  put  a  sterile  dressing  over  the  catheter  site. 

Research 

Support 

Pronovost  and  colleagues  pilot-tested  the  checklist  in  2004  in  one  surgical  ICU  at  Johns 
Hopkins  Hospital  and  found  its  use  virtually  eliminated  catheter-related  bloodstream 
infections  among  patients  there  (Berenholtz  et  al.,  2004).  This  efficacy  study  then  led  to 
two  large-scale  effectiveness  studies  in  which  the  checklist  was  tested  in  108  hospitals 
in  Michigan  (Pronovost  et  al.,  2006).  Thus,  use  of  the  checklist  expanded  beyond  a  single 
site  to  widespread  practice  and  evaluation.  Researchers  found  that  rates  of  infection 
were  reduced  by  66  percent  during  the  time  the  checklist  was  used  in  the  hospitals.  A 
follow-up  study  in  Rhode  Island  found  that  infection  rates  fell  by  74  percent  across  the 
state's  23  ICUs  over  a  period  of  two  and  a  half  years  (Lashoher  and  Pronovost,  2010). 
Researchers  estimated  that  the  checklist  helped  reduce  hospital  stays  by  a  cumulative 

608  days  and  saved  approximately  $2  million  in  health  care  costs. 

Key  Materials 

A  checklist  posted  in  many  hospitals  in  ICUs  and  surgical  operating  rooms. 

Status  of 

Diffusion 

The  checklist  approach  has  now  been  widely  adopted  at  many  hospitals  throughout  the 
country. 

Key  Components 
of  Diffusion 

Evidence  of  real-world  effectiveness:  Evidence  of  quality  research  preceding 
effectiveness  studies;  efficacy  study  at  single  site  used  to  support  wider  implementation 
across  several  sites  in  effectiveness  studies 

Evidence  of  an  identified  need:  Simplicity  of  implementation  (i.e.,  checklist  format); 
demonstration  of  cost  savings  and  saving  of  lives 

Leadership  buy-in  and  support  from  key  stakeholders:  Leadership  support  from  DHHS 

Collaboration  with  credible  sponsors;  Partnerships  with  medical  departments, 
hospitals,  and  other  organizations 

Provision  of  incentives  or  development  of  policies:  Ease  of  implementation  in  hospital 
settings;  standardizes  an  approach  providers  already  use 

Peer  network  supportive  of  adoption:  Training  and  buy-in  from  providers;  peer  support 
and  accountability  (e.g.,  nurses  can  remind  doctors  to  follow  the  checklist  during 
procedures) 

7.  Leaders  can  demonstrate  the  need  for  an  innovation  by  presenting  data  or  supporting  atten¬ 
dance  at  conferences  by  adopters  on  the  ground.  Leaders  can  work  with  researchers  to  dem¬ 
onstrate  the  need  for  an  innovation  to  other  leaders,  policymakers,  and  adopters  on  the 
ground. 

8.  Senior  leadership  can  help  increase  coordination  and  synchronization  among  departments, 
disciplines,  and  organizations.  Because  effective  diffusion  necessitates  joint  efforts  among 
individuals  and  systems,  leaders  can  help  develop  and  manage  the  relationships  needed 
for  successful  adoption. 
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The  role  of  senior  management  is  crucial  in  successfully  adopting  research  into  an  orga¬ 
nization’s  structure  (Bradley  et  ah,  2004;  Yuan  et  ah,  2010).  Leaders  are  uniquely  positioned 
to  help  facilitate  the  adoption  of  a  program  by  supporting  adopters  on  the  ground  prior  to, 
during,  and  after  the  implementation  phases.  Successful  case  studies  documenting  reorganiza¬ 
tion  within  large  health  care  systems  attribute  strong  leadership  to  the  successful  implementa¬ 
tion  of  change  (Lukas  et  ah,  2007).  In  sharing  ideas  across  organizations,  leaders  can  generate 
novel  strategies  for  the  diffusion  of  research  findings  among  adopters.  Cross-organization  col¬ 
laboration  can  also  inform  leaders  about  what  may  actually  work  on  the  ground. 

Specific  to  the  military  population,  the  Air  Force  Suicide  Prevention  Program  has  dem¬ 
onstrated  effectiveness  in  reducing  the  relative  risk  of  suicide  among  service  members  exposed 
to  a  program  with  strong  leadership  support  (Knox,  Litts,  et  ah,  2003).  Leadership  support 
included  continued  encouragement  of  senior  leaders  to  promote  protective  factors  for  reduced 
suicide  risk  and  urging  personnel  in  need  of  help  to  seek  resources  early  on.  Utilizing  many  of 
the  components  outlined  in  this  chapter,  including  continuous  evaluation  after  implementa¬ 
tion  of  a  new  program,  the  program  has  demonstrated  effectiveness  in  reducing  suicide  risk  in 
the  Air  Force  since  its  implementation  (Knox,  Pflanz,  et  ah,  2010). 

Box  6.5  presents  one  case  study  that  illustrates  this  fourth  component:  Project  HOPE, 
Hawaii’s  Opportunity  Probation  with  Enforcement. 

5.  Funding  or  Other  Institutional  Support 

Any  program  needs  some  degree  of  funding  to  be  successful  in  practice  (Greenhalgh  et  ah, 
2004).  Here,  we  address  the  various  types  of  necessary  funding. 

Continued  Research  and  Evaluation 

The  third  component  outlined  in  this  chapter  concerns  the  logical  final  step  after  a  series  of 
efficacy  studies:  developing  an  effectiveness  study.  Tkese  studies  can  be  costly  and  require 
a  large  amount  of  resources.  TTius,  most  studies  supported  by  efficacy  research  do  not  get 
implemented  into  practice  (Glasgow,  Eichtenstein,  and  Marcus,  2003).  Eunding  agencies  can 
offer  additional  opportunities  or  incentives  for  effectiveness  research,  which  may  increase 
the  number  of  effectiveness  study  trials  on  specific  topics  of  interest.  Eor  example,  Datillio, 
Edwards,  and  Eishman  (2010)  describe  a  mixed-methods  paradigm  that  involves  a  series  of 
parallel  studies  within  a  broader  research  project  (see  Appendix  E  for  further  details).  Tliese 
studies  include  RGTs  complemented  by  project  implementation  and  qualitative  data  collec¬ 
tion,  followed  by  systematic  case  studies  once  the  project  has  been  implemented.  Having  tar¬ 
geted  funding  opportunities  that  build  in  an  effectiveness  component  may  encourage  more 
research  of  this  type. 

Eunding  agencies  can  also  offer  specific  funding  to  evaluate  preexisting  programs  that 
may  have  little  empirical  support.  Thus,  it  is  becoming  increasingly  important  for  funding 
mechanisms  to  promote  effectiveness  research  (or  research  that  can  applied  to  real-world  popu¬ 
lations)  so  that  findings  can  be  applicable  on  a  broader  and  more  generalizable  scale  (Bradley 
et  ah,  2004;  Yuan  et  ah,  2010). 

Incentivizing  Adoption 

Eunding  specified  to  help  organizations  adopt  research-based  programs  could  be  made  avail¬ 
able.  Eor  example,  SAMHSA  offers  discretionary  funding  accompanied  by  technical  support 
to  promote  the  adoption  of  evidence-based  practices  (Gotham,  2006).  Another  example  is 
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Box  6.5 

Case  Study:  Project  HOPE,  Hawaii's  Opportunity  Probation  with  Enforcement 


Identified  Need 

Like  many  other  states,  Hawaii  was  experiencing  a  problem  with  its  probation  system. 
Convicted  felons  were  failing  to  present  for  probation  hearings  and  were  declining  to 
take  mandatory  drug  tests.  Despite  this,  the  police,  parole  officers,  and  the  courts  were 
failing  to  take  action  against  these  infractions. 

Innovation 

In  2004,  Circuit  Judge  Steven  Aim  garnered  support  from  the  police,  local  jail  officials, 
parole  officers,  drug  treatment  professionals,  prosecutors,  and  deference  counsel  to 
create  Project  HOPE  (Hawaii's  Opportunity  Probation  with  Enforcement),  which  would 
provide  "swift  and  certain"  penalties  for  parolees  violating  the  conditions  of  their 
parole.  Felons  were  told  at  their  initial  hearings  that  probation  would  be  more  strictly 
enforced  with  penalties.  It  was  hypothesized  that  certainty  and  swiftness  would  be 
more  effective  than  severity.  Project  HOPE  features  (1)  a  focus  on  reducing  drug  use  and 
missed  appointments  rather  than  imposing  drug  treatment  on  all  felons,  (2)  reduced 
costs  and  less  resource  usage  by  having  probationers  appear  in  court  only  if  a  violation 
is  detected  and  by  mandating  drug  treatment  only  for  repeated  failed  drug  tests, 

(3)  reduced  costs  to  society  by  having  probationers  who  are  employed  serve  jail  time  on 
the  weekends  so  they  can  continue  their  employment,  and  (4)  reduced  length  of  stay  in 
prison  and  in  treatment  centers  for  first-time  offenders  so  that  those  who  need  care  the 
most  can  receive  treatment  in  residential  treatment  centers. 

Research 

Support 

Project  HOPE  leaders  teamed  with  researchers  at  the  University  of  California,  Los 

Angeles,  and  Pepperdine  University  to  empirically  evaluate  the  effects  of  the  program. 

An  RCT  was  conducted  in  Hawaii  to  examine  reductions  in  drug  use  and  missed 
appointments  and  to  determine  the  cost-effectiveness  of  the  program.  The  NIH  funded 
the  trial,  which  included  nearly  500  men  and  women  identified  as  being  at  risk  for 
parole  violations  (e.g.,  methamphetamine  users,  sex  offenders,  domestic  violence 
offenders).  Two-thirds  participated  in  Project  HOPE,  while  the  other  one-third  received 
probation  as  usual.  After  one  year,  compared  with  a  control  group.  Project  HOPE 
probationers  were  55  percent  less  likely  to  be  arrested  for  a  new  crime,  72  percent  less 
likely  to  use  drugs,  61  percent  less  likely  to  skip  appointments,  and  53  percent  less  likely 
to  have  their  probation  revoked.  Project  HOPE  probationers  also  served  less  time  in  jail. 

Key  Materials 

Project  HOPE'S  studies  are  available  online.  The  widespread  adoption  of  policies  at 
the  state  level  can  be  applied  to  jurisdictions  through  written  policies  describing  the 
approach. 

Status  of 

Diffusion 

Today,  more  than  3,000  probationers  (about  40  percent  of  convicted  felons)  in  Hawaii 
participate  in  Project  HOPE  (Aim,  2010).  Project  HOPE  is  now  being  evaluated  for  long¬ 
term  effectiveness  (five  years  post-participation).  Four  other  states  have  used  Project 

HOPE:  Oregon,  Massachusetts,  Texas,  and  Arkansas.  The  Research  Triangle  Institute  has 
paired  with  Pennsylvania  State  University  to  continue  evaluating  further  outcomes  of 
the  program. 

Key  Components 
of  Diffusion 

Leadership  buy-in  and  support  from  key  stakeholders:  Strong  senior  leadership  within 
the  courts;  support  from  senior  leadership  with  courts  and  at  the  state  level  created 
a  collaborative  effort  among  the  courts,  probation  officers,  police,  drug  treatment 
centers,  and  researchers;  successful  judges/courts  served  as  role  models,  and  Hawaii 
serves  as  a  role  model  for  other  state  jurisdictions 

Evidence  of  an  identified  need:  Fit  the  needs  of  an  identified  problem  with  the  system 
in  use 

Evidence  of  research  quality:  Strong  empirical  support;  continued  evaluation 

Evidence  of  real-world  effectiveness:  Studies  in  development  in  several  states 

Collaboration  with  credible  sponsors:  Collaborative  research  effort;  courts  paired  with 
university  researchers  and  assessed  outcomes  and  cost-effectiveness 

Peer  network  supportive  of  adoption:  Buy-in  from  several  jurisdictions 
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the  Patient- Centered  Outcomes  Research  Institute,  an  independent,  nonprofit  health  research 
organization  established  in  2010  under  the  Patient  Protection  and  AfFordahle  Care  Act.  The 
institute’s  research  agenda  involves  funding  projects  that  seek  to  facilitate  the  use  of  research 
evidence  to  meet  the  needs  of  patients,  caregivers,  and  health  care  professionals.  Thus,  effec¬ 
tiveness  research  is  made  a  priority  up  front  in  funding  announcements  and  preference  is  given 
to  projects  that  attempt  to  make  research  knowledge  available  to  patients  and  providers  in  a 
timely  manner.  This  can  involve  sharing  knowledge  of  research  hypotheses  and  strategies  with 
those  on  the  ground  at  the  initial  phase  of  the  research,  rather  than  as  an  afterthought  once 
the  study  is  completed.  This  approach  helps  facilitate  mutual  decisionmaking  between  care 
providers  and  patients. 

Additional  forms  of  support  are  necessary  for  successful  diffusion,  including  managed- 
care  reimbursements  to  encourage  broad-scale  implementation  across  sites  (e.g.,  Lashoher  and 
Pronovost,  2010;  Swenson,  2000).  Reimbursing  hospitals,  clinics,  and  installations  for  using 
empirically  supported  programs  can  provide  further  incentives  for  organizations  to  learn  about 
and  adopt  these  programs.  For  example,  managed-care  companies  support  the  use  of  DBT  for 
hospital  patients  with  BPD.  This  has  been  attributed  as  a  major  factor  supporting  the  wide¬ 
spread  adoption  of  DBT  among  providers. 

Support  Infrastructure 

Programs  need  continued  dedicated  support  to  keep  running.  Support  infrastructure  is  a  key 
element  of  diffusion,  such  as  money  to  support  and  incentivize  staff,  to  keep  program  materials 
current  and  suitable  for  dissemination  among  targeted  individuals,  and  to  support  the  technol¬ 
ogy  or  media  necessary  to  continue  the  program.  It  may  be  necessary  to  hire  dedicated  staff  to 
run  the  program,  and  an  approach  that  has  been  successful  in  helping  new  programs  continue 
to  meet  their  goals  (see,  e.g.,  Daschner  et  ah,  2000;  Lapane  et  ah,  2011).  Dedicated  funding 
that  is  used  for  program  needs  only  can  also  help  ensure  that  there  is  adequate  financial  sup¬ 
port  to  keep  the  program  active.  Leadership  can  help  in  obtaining  and  sustaining  the  resources 
needed. 

Box  6.6  presents  one  case  study  that  illustrates  this  fifth  component:  Project  ALERT. 

6.  Collaboration  with  Credible  Sponsors 

The  process  of  research  diffusion  is  slowed  when  significant  effort  is  required  to  coordinate 
across  departments  or  disciplines  within  organizations  (Bradley  et  ah,  2004).  Similarly,  the 
adoption  of  research-based  practices  can  be  slowed  if  there  is  not  already  an  established  work¬ 
ing  relationship  between  the  dissemination  infrastructure  and  the  adopting  organizations 
(Bradley  et  ah,  2004;  Yuan  et  ah,  2010). 

It  can  be  challenging  to  garner  support  from  personnel  and  create  cohesive  care  teams 
across  multiple  departments  and  disciplines.  Thus,  forming  a  coalition  of  credible  sponsors  is 
important  for  successful  diffusion.  Coalitions  of  federal,  national,  and  state  organizations  have 
increased  the  credibility  of  practices  and  helped  raise  awareness  of  campaigns  (Berwick,  2003; 
Berwick  et  ah,  2006;  Wachter  and  Pronovost,  2006).  Programs  that  have  successfully  dif¬ 
fused  research  into  practice  have  the  breadth  and  geographic  diversity  of  the  partnering  orga¬ 
nizations  that  support  them  (Yuan  et  ah,  2010).  If  “ownership”  of  a  program  is  shared  across 
many  departments,  commitment  to  the  program  across  an  organization  can  be  increased. 
Indeed,  improvement  initiatives  that  actively  engage  staff  in  change  decisions  and  involve  staff 
at  all  organizational  levels  in  developing  goals  are  successful  components  of  effective  system- 
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Box  6.6 

Case  Study:  Project  ALERT 


Identified  Need 

Middle  school  and  high  school  are  at-risk  periods  for  the  initiation  of  alcohol  and  other 
drug  abuse.  There  have  been  multiple  drug  prevention  programs  for  students  at  these 
grade  levels  but  very  few  with  any  empirical  support. 

Innovation 

Project  ALERT  was  established  in  1984  as  a  school-based  alcohol  and  drug  use 
prevention  program  for  seventh-  and  eighth-grade  students.  It  was  developed  with 
a  focus  on  early  prevention  to  provide  knowledge  about  drugs'  effects  and  to  teach 
drug  refusal  skills  (e.g.,  overcoming  social  pressure).  Project  ALERT  trained  teachers 
to  deliver  a  multisession  curriculum  to  students  in  their  classrooms.  The  program  has 
been  endorsed  by  several  agencies  with  their  highest  levels  of  support.  For  example. 
Project  ALERT  was  endorsed  by  SAMHSA  in  1999  as  a  "model  program,"  and  the  U.S. 
Department  of  Education  gave  Project  ALERT  its  "exemplary  program"  rating  in  2001. 

The  program  also  has  a  "proven  effectiveness"  rating  from  the  Promising  Practice 

Network  on  Children,  Families  and  Communities. 

Research 

Support 

The  program  of  research  followed  by  the  Project  ALERT  team  is  a  model  for  diffusing  a 
program  using  pilot-testing  and  efficacy  studies,  effectiveness  studies,  and  continued 
evaluation  (e.g.,Ellickson  and  Bell,  1990;  Ellickson,  Bell,  and  McGuigan,  1993;  Ellickson, 
Tucker,  and  Klein,  2003;  Tucker,  Ellickson,  and  Klein,  2002).  The  program  was  pilot- 
tested  using  an  efficacy  study  in  the  mid-1980s  at  30  middle  and  junior  high  schools 
in  California  and  Oregon.  Compared  with  a  control  group.  Project  ALERT  students 
used  cigarettes  and  marijuana  less  frequently.  Using  findings  from  the  efficacy  study 
to  design  the  program  for  diffusion,  materials  were  standardized  and  tested  for 
effectiveness  beginning  in  1995.  More  than  4,000  students  in  55  middle  schools  in 

South  Dakota  participated  in  the  program;  compared  with  controls,  the  program 
demonstrated  preventive  effects  on  alcohol  and  drug  use  over  two  years. 

Key  Materials 

Project  ALERT  researchers  created  a  website  that  offers  free,  interactive  training  with 
videos,  published  materials  (e.g.,  manuals,  posters),  and  practice  scenarios.  Materials 
are  also  available  in  Spanish.  The  training  is  available  to  teachers  and  facilitators  from  a 
variety  of  backgrounds  in  diverse  settings.  The  website  offers  training  both  prior  to  and 
after  implementation  of  the  program,  with  fidelity  measures  and  feedback  to  support 
quality  assurance. 

Status  of 

Diffusion 

Project  ALERT  has  become  a  well-known  and  well-diffused  alcohol  and  drug  prevention 
program  used  by  more  than  3,500  school  districts  in  every  state.  More  than  18,000 
classroom  teachers  are  trained  to  deliver  the  14-lesson  curriculum.  Current  research 
efforts  include  four-  and  five-year  outcome  studies  of  the  program  in  several  middle 
schools  throughout  the  country,  as  well  as  an  empirical  study  of  the  added  benefit  of 
implementing  booster  sessions  for  students  once  they  are  in  high  school. 

Key  Components 
of  Diffusion 

Funding  or  other  institutional  support:  Endorsement  by  SAMHSA  and  the  U.S. 

Department  of  Education;  National  Institute  on  Drug  Abuse  and  foundation  support; 
dedicated  funding  to  sustain  the  program 

Evidence  of  an  identified  need:  Empirically  supported  prevention  efforts  needed  for 
adolescents 

Evidence  of  research  quality:  Strong  program  of  research  efficacy;  continued  evaluation 

Evidence  of  real-world  effectiveness:  Implementation  in  more  than  3,500  school 
districts  throughout  the  country 

Dissemination  materials:  Accessible,  free  online  training  and  materials;  fidelity  checks 
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wide  change  in  the  organizational  structure  (Lukas  et  ah,  2007).  Leadership  can  assist  in  this 
effort  hy  promoting  innovations  while  modeling  and  incentivizing  successful  collahoration 
between  and  within  organizations.  Establishing  collaborations  among  an  organization’s  leader¬ 
ship  (including  senior  leaders  and  key  stakeholders),  researchers,  and  individuals  on  the  ground 
creates  a  joint  effort  supporting  an  identihed  target  (Bradley  et  ah,  2004;  Yuan  et  ah,  2010). 

Research  institutions  are  uniquely  positioned  to  understand  how  efhcacy  and  effective¬ 
ness  studies  work,  and  partnering  with  academic  researchers  can  help  inform  efforts  on  the 
ground.  Accounting  for  the  unique  perspectives  of  providers,  line  leaders,  and  adopters  is  also 
essential  to  understanding  how  an  innovation  may  be  implemented  into  practice.  Organiza¬ 
tions,  decisionmakers,  and  target  audiences  can  all  be  involved  in  the  development  of  efhcacy 
studies  in  the  planning  stages.  Researchers  can  also  work  with  community  partners  to  research 
ideas  that  are  relevant  to  their  organization.  For  example,  providers  in  the  held  with  military 
personnel  can  share  their  ideas  with  researchers  about  what  they  believe  works  based  on  anec¬ 
dotal  experience.  This  approach  also  helps  study  designers  better  understand  the  needs  of  the 
system  they  are  working  to  improve  (e.g.,  making  sure  to  include  enough  ethnic  minority 
participants  in  a  sample  to  make  the  sample  generalizable  to  the  organization’s  demographics). 
Researchers  can  then  design  studies  to  identify  why  a  particular  approach  may  be  efficacious 
and  involve  the  community  in  all  levels  of  the  experimental  design  process.  This  can  also  help 
facilitate  data  collection  and  promote  more  enthusiasm  for  findings,  because  community  mem¬ 
bers  have  an  investment  in  the  research. 

Box  6.7  presents  a  case  study  that  illustrates  this  sixth  component:  nail  gun  safety. 

7.  Provision  of  Incentives  or  Development  of  Policies 

Providers  are  open  to  learning  about  findings  from  research  studies  and  generally  express  posi¬ 
tive  opinions  about  research  accomplishments  (Cook,  Biyanova,  and  Coyne,  2009;  Stewart, 
Stirman,  and  Chambless,  2012),  but  they  often  do  not  learn  about  research  strategies  because 
of  limited  time,  costly  training,  or  a  lack  of  opportunity  (Pagoto,  2007;  Riley  et  ah,  2007). 
There  are  also  limited  resources  and  incentives  for  providers  to  learn  about  research.  In  addi¬ 
tion  to  providers,  nonclinical  support  staff  and  adopters  on  the  ground  (e.g.,  chaplains,  sub¬ 
stance  abuse  program  counselors,  transition  assistance  advisers,  suicide  prevention  coordina¬ 
tors)  may  not  have  the  expectation  to  publish  or  the  continuing  education  requirements  that 
would  incentivize  them  to  stay  current  on  research  and  evaluation.  In  the  military,  front-line 
leaders  and  personnel  working  closely  with  service  members  may  not  have  the  time,  flexibility, 
or  interest  to  review  journal  articles  and  technical  reports.  Personnel  who  do  not  specialize  in 
suicide  prevention  may  not  be  fully  aware  of  the  latest  developments  in  the  suicide  prevention 
field. 

It  is  important  that  those  who  are  interested  in  diffusing  research  studies  into  practice 
incentivize  those  on  the  ground  to  implement  the  targeted  programs.  This  can  be  done  by  pro¬ 
viding  incentives  and  developing  policies  to  ensure  smooth  transition  of  a  research  program 
into  new  settings.  Here,  we  review  some  common  types  of  incentives. 

Continuing  Education 

Providers  can  be  given  more  educational  opportunities,  such  as  a  chance  to  attend  conferences 
(or  a  requirement  to  do  so).  There  could  be  financial  incentives  for  learning  about  new  research 
strategies,  such  as  a  gift  card  for  providers  who  attend  an  on-site  research  talk  that  would 
typically  be  attended  by  researchers  only.  It  may  be  helpful  to  have  a  data  interpreter  who  can 
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Box  6.7 

Case  Study:  Nail  Gun  Safety 


Identified  Need 

Nail  gun  injuries  account  for  about  40,000  emergency  department  visits  each  year,  with 

60  percent  among  construction  workers.  The  rest  of  the  injuries  (40  percent)  are  among 
non-construction  worker  consumers  (Center  for  Construction  Research  and  Training, 

2012). 

Innovation 

The  National  Institute  for  Occupational  Safety  and  Health  (NIOSH)  has  developed  a 
policy  program  called  Research  2  Practice,  which  sponsors  research  on  several  issues 
related  to  safety  and  health  among  workers.  Research  2  Practice  also  helps  disseminate 
practical  findings  about  how  to  reduce  risks.  One  of  these  issues  is  nail  gun  safety, 
which  is  a  collaborative  research  and  awareness  effort  fronted  by  the  Center  for 
Construction  Research  and  Training. 

Research 

Support 

The  Center  for  Construction  Research  and  Training  designed  a  series  of  multiyear 
studies  in  a  collaborative  research  effort  with  Duke  University  researchers,  carpenters 
in  St.  Louis,  and  regional  homebuilders  associations  to  determine  the  extent  of  nail 
gun  injuries  and  the  safest  type  of  nail  gun.  The  researchers  asked  more  than  2,000 
construction  workers  to  complete  a  written  questionnaire  about  the  nail  guns  they 
used.  They  then  conducted  in-depth  interviews  with  a  subsample  of  500  of  workers 
who  had  reported  a  nail  gun  injury.  Study  participants  also  received  training  on  the 
safe  use  of  nail  guns.  Findings  from  the  studies  revealed  that  sequential-trigger  nail 
guns  (i.e.,  on  which  the  nose  of  the  gun  must  be  pushed  in  before  trigger  can  be  pulled) 
were  safer  than  contact-tip  nail  guns  (i.e.,  those  that  fire  when  the  trigger  is  squeezed 
and  the  tip  touches  something,  also  known  as  "bump  nailers").  The  researchers  found 
that  the  risk  of  injury  was  double  for  contact-tip  nail  guns,  with  a  55-percent  reduction 
in  the  risk  of  injury  when  using  sequential-trigger  nail  guns.  Construction  workers  and 
employees  selling  nail  guns  were  unaware  of  these  risks  and  were  not  properly  trained 
in  nail  gun  safety  (Lipscomb  et  al.,  2011). 

Key  Materials 

Study  findings  and  the  dissemination  efforts  of  Research  2  Practice  led  to  the  creation 
of  several  fact  sheets  on  nail  gun  safety  (Center  for  Construction  Research  and  Training, 
2012),  guidance  documents  (e.g..  Nail  Gun  Safety:  A  Guide  for  Construction  Contractors, 
published  by  the  NIOSH  and  the  Occupational  Safety  and  Health  Administration  [2011]), 
and  an  interactive  website  with  safety  tools  and  guidelines. 

Status  of 

Diffusion 

A  nail  gun  safety  website  has  had  more  than  250,000  unique  visits  since  2011.  Trainings 
were  developed  to  reduce  the  risk  of  injury  among  construction  workers  and  employees 
selling  nail  guns. 

Key  Components 
of  Diffusion 

Collaboration  with  credible  sponsors:  Commitment  from  a  multiyear  initiative  from 
academic  researchers,  labor,  and  management;  guidelines  generated  for  guidance 
documents  incorporated  feedback  from  research  participants  (including  those  who  had 
suffered  an  injury)  and  experts 

Evidence  of  research  quality:  Researchers  continue  to  conduct  research  and  share  their 
findings  with  key  government  agencies,  insurance  companies,  and  consumers  to  spread 
the  message  about  nail  gun  safety 

Evidence  of  an  identified  need:  Nail  gun  injuries  were  sending  more  than  40,000  people 
to  emergency  departments  each  year 

Evidence  of  real-world  effectiveness:  Evaluation  of  knowledge  and  awareness  of  nail 
gun  safety 

Leadership  buy-in  and  support  from  key  stakeholders:  Leaders  at  NIOSH  headed  the 
effort 

Funding  or  other  institutional  support:  Sponsors  interested  in  funding  the  program 

Peer  network  supportive  of  adoption:  Training  for  construction  workers  and  store 
employees  in  nail  gun  safety 

Dissemination  materials:  Fact  sheets,  interactive  website,  guidance  documents 
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serve  as  a  point  person  between  a  research  setting  (e.g.,  the  university)  and  the  practice  setting 
(e.g.,  the  university-affiliated  hospital)  who  can  summarize  efficacy  and  effectiveness  research 
findings  on  a  regular  basis  through  briefings,  presentations,  or  easy-to-follow  emailed 
summaries. 

Incentivizing  Adoption 

As  discussed  in  connection  with  the  fifth  component  earlier  in  this  chapter,  funders  can  con¬ 
tribute  to  diffusion  efforts  by  funding  projects  that  seek  to  facilitate  the  use  of  research  evi¬ 
dence  in  practice  (effectiveness  research).  Effectiveness  research  can  be  specified  up  front  in 
funding  announcements,  with  priority  given  to  projects  that  incorporate  transferring  informa¬ 
tion  obtained  from  the  research  study  (e.g.,  what  is  effective  and  for  whom)  to  patients  and 
providers. 

Guidelines 

If  providers  determine  that  guidelines  for  care  are  too  complex,  they  are  typically  less  likely 
to  follow  them  (Davis  and  Taylor-Vaisey,  1997;  Francke  et  ak,  2008;  Lomas,  1993).  There  is  a 
greater  chance  that  a  guideline  will  be  carried  out  when  it  is  easily  understood,  straightforward, 
and  relatively  simple  implement  (Rogers,  2003;  Yuan  et  ak,  2010).  However,  this  can  prove  dif¬ 
ficult  when  creating  guidelines  for  use  by  multidisciplinary  groups,  which  may  have  varying 
levels  of  expertise  and  knowledge. 

In  the  health  care  field,  clinical  practice  guidelines  (CPGs)  have  been  created  to  help 
bridge  the  gap  between  research  and  practice.  CPGs  can  be  seen  as  facilitators  of  policy  changes 
because  they  were  first  created  after  system-wide  policy  changes  were  met  with  resistance  from 
providers.  CPGs  are  based  on  systematic  literature  reviews  and  recommendations  by  experts 
and  are  intended  to  guide  practitioners  through  a  series  of  decisions  when  implementing  a 
treatment  plan  (Anderson  et  ak,  2002;  Field  and  Lohr,  1990).  CPG  implementation  is  fol¬ 
lowed  by  dissemination,  which  is  a  multistage  process  that  involves  making  practitioners  aware 
of  guidelines  and  tracking  whether  they  are  followed.  It  is  no  surprise  that  providers  cannot 
follow  a  clinical  practice  guideline  if  they  are  not  aware  of  its  existence,  even  after  widespread 
dissemination  (Cabana  et  ak,  1999). 

An  example  of  a  CPG  is  the  VA/DoD  Clinical  Practice  Guideline  for  Assessment  and  Man¬ 
agement  of  Patients  at  Risk  for  Suicide  (VA  and  DoD,  2013),  which  outlines  a  series  of  steps 
clinicians  can  take  if  a  patient  expresses  suicidal  ideation  or  meets  a  specific  threshold  for  rec¬ 
ommended  care  on  a  self-reported  measure  of  behavior.  It  focuses  on  standardizing  practices 
across  the  VA  and  DoD  to  help  improve  patient  outcomes  while  providing  evidence-based 
recommendations  to  help  providers  make  decisions  about  how  to  assess  and  treat  patients  at 
risk  for  suicide. 

In  sum,  given  time  constraints  and  varying  levels  of  interest  in  research  findings,  it  may 
be  necessary  to  provide  more  incentives  and  educational  opportunities  for  those  on  the  ground 
who  are  expected  to  adopt  a  new  program.  CPGs  bridge  the  gap  between  research  and  practice 
and  help  with  adoption  after  policy-level  changes  occur  within  an  organization. 

Box  6.8  presents  one  case  study  that  illustrates  this  seventh  component:  evidence-based 
practices  for  PTSD  in  the  Veterans  Health  Administration  (VHA). 
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Box  6.8 

Case  Study:  Evidence-Based  Practices  for  PTSD  in  the  Veterans  Health  Administration 


Identified  Need 

Increasing  numbers  of  veterans  were  seeking  care  for  PTSD  through  the  VHA.  The  VHA 
was  not  utilizing  the  available  empirically  supported  interventions  for  PTSD  treatment. 

The  standard  of  care  was  lengthy,  costly,  and  not  evaluated  for  effective  outcomes. 

Innovation 

DoD  partnered  with  the  VHA  to  implement  a  large-scale  rollout  of  two  cognitive 
behavioral  treatments  for  PTSD  with  the  greatest  degree  of  research-supported  efficacy 
(prolonged  exposure  and  cognitive  processing  therapy;  Karlin  et  al.,  2010).  The  effort 
to  train  thousands  of  providers,  provide  consultations,  and  evaluate  outcomes  was 
made  possible  by  the  VA/DoD  CPG  for  PTSD  (VA  and  DoD,  2010),  which  stated  that  the 
interventions  are  always  indicated  and  appropriate  when  treating  veterans  for  PTSD. 

Research 

Support 

The  VHA  effectively  rolled  out  the  therapies  (Karlin  et  al.,  2010)  and  more  than  4,000 

VA  and  DoD  mental  health  staff  have  been  trained  (Chard  et  al.,  2012).  There  is  a  large 
research  base  supporting  both  prolonged  exposure  and  cognitive  processing  therapy 
with  civilian  and  veteran  samples. 

Key  Materials 

Trainers  and  resources  are  available  in  VA  settings,  including  a  point  person  for  each 
region  who  trains  providers  in  the  approaches  and  training  other  providers  in  the 
system.  Regular  consultation  is  offered,  including  conference  calls  with  intervention 
developers,  and  providers  are  given  incentives  for  participating  in  the  trainings 
(e.g.,  becoming  a  certified  cognitive  processing  therapy  provider,  which  can  help 
with  job  opportunities  in  the  future). 

Status  of 

Diffusion 

With  the  aid  of  leadership  and  increased  funding  for  positions  such  as  evidence-based 
psychotherapy  coordinators  and  a  PTSD  mentoring  program,  the  VHA  is  now  a  leader  in 
effective  PTSD  treatment  and  a  model  for  PTSD  care  throughout  the  country. 

Key  Components 
of  Diffusion 

Provision  of  incentives  or  development  of  policies:  Incentives  offered  throughout 
dissemination  process;  VHA-wide  policy  change  to  offer  prolonged  exposure  or 
cognitive  processing  therapy  for  all  PTSD  cases;  CPGs  accompanied  by  trainings, 
consultations,  and  mentorship 

Evidence  of  an  identified  need:  Diagnoses  of  PTSD  among  service  members  returning 
from  Iraq  and  Afghanistan;  recognized  need  for  empirically  based  interventions 

Evidence  of  research  quality;  Rollout  of  two  interventions  cited  as  efficacious  in 
multiple  research  studies  with  civilians  and  veterans 

Leadership  buy-in  and  support  from  key  stakeholders;  Leadership  at  both  the  federal 
and  local  levels  mandated  training  in  prolonged  exposure  and  cognitive  processing 
therapy;  consultation  opportunities 

Funding  or  other  institutional  support:  Dedicated  funding  for  trainers,  mentors,  and 
point  person  on  site 

Collaboration  with  credible  sponsors:  VA-DoD  partnership 

Peer  network  supportive  of  adoption:  Decentralized  training  in  which  trainees  become 
trainers,  promoting  widespread  diffusion;  supervisors  train  interns  and  practicum 
students 

8.  Peer  Network  Supportive  of  Adoption 

The  importance  of  peers  in  disseminating  research  findings  cannot  be  overestimated.  Health 
care  providers  learn  about  innovations  through  informal  channels  and  sometimes  cannot  even 
identify  where  they  first  learned  about  a  new  practice  (Miller  et  ah,  2006;  Rosen,  2000).  They 
also  typically  guide  their  treatment  decisions  based  on  experience  or  consultation  with  others. 
Studies  have  documented  that  health  care  providers  are  generally  more  comfortable  with  infor- 
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mation  they  receive  from  colleagues  than  what  they  find  in  research  articles  or  hooks  (Cohen, 
2007;  Cohen,  Sargent,  and  Sechrest,  1986;  Morrow-Bradley  and  Elliott,  1986),  and  they  use 
patient  information  rather  than  research  to  inform  treatment  plans  (Dattilio,  Edwards,  and 
Fishman,  2010;  Stewart  and  Chamhless,  2007;  Stewart,  Stirman,  and  Chamhless,  2012). 
Rogers  (2003)  refers  to  “ohservahility”:  the  extent  to  which  the  benefits  of  a  new  practice  can 
be  observed  by  others. 

Although  providers  are  interested  in  learning  about  innovative  research  strategies, 
they  generally  rely  on  their  peers  for  information  about  research-informed  practices  (Cook, 
Biyanova,  and  Coyne,  2009;  Stewart,  Stirman,  and  Chamhless,  2012).  Consultation  with  cred¬ 
ible  peers  may  be  an  important  avenue  for  the  diffusion  of  research  into  practice  (Tan,  2002).  It 
is  likely  that  similar  practices  exist  among  non-health  care  providers  who  may  be  key  players 
in  diffusing  research  (i.e.,  installation  leaders,  chaplains,  or  even  noncommissioned  officers). 

Providers  and  those  on  the  ground  who  are  implementing  the  research  into  practice  are 
perhaps  the  most  important  targets  of  diffusion  efforts.  Thus,  it  is  essential  to  understand  how 
diffusion  occurs  through  peer  networks  among  these  individuals.  Many  times,  it  takes  a  few 
early  adopters  of  an  innovation  to  champion  diffusion  (Rogers,  2003),  because  these  individu¬ 
als  promote  the  practice,  model  for  others,  and  reinforce  successful  adoption  for  their  peers. 

As  research  programs  are  diffused  into  practice,  the  process  of  adoption  typically  follows 
an  S-shaped  curve  (Rogers,  2003).  Diffusion  typically  begins  at  a  relatively  flat  rate,  with  a  few 
innovative  providers  and  opinion  leaders  in  the  early  adoption  phases.  The  curve  turns  upward 
with  a  steep  incline  as  the  early  majority  adopts  the  innovation.  The  S-curve  begins  to  level 
out  and  will  plateau  as  skeptical  late  adopters  eventually  adopt  the  innovation.  The  pattern 
reveals  that  it  is  important  for  leaders  and  a  small  group  of  providers  to  adopt  the  innovation 
in  the  beginning  stages,  to  be  seen  adopting  the  innovation  by  peers,  and,  in  turn,  to  influence 
the  eventual  adoption  of  the  practice  over  time.  Rogers  (2003)  has  identified  five  categories  of 
innovation  adopters  (see  Table  6.2). 


Table  6.2 

Rogers'  Categories  of  Innovation  Adopters 


Adopter  Type 


Description 


Innovators  Venturesome  individuals  who  tend  to  be  the  earliest  adopters.  These  individuals  tend  to  be  the 
ones  pushing  for  change  and  have  the  financial  resources  and  connections  to  institute  change. 
They  have  the  ability  understand  the  complexity  of  a  new  idea  and  to  cope  with  the  uncertainty 
that  may  accompany  it.  These  individuals  require  a  shorter  adoption  period  than  those  in  any 
other  category. 

Early  adopters  Opinion  leaders  who  try  out  an  innovation  in  a  careful  manner.  These  individuals  are  respected 
by  their  peers,  and  they  become  a  source  of  modeling  and  advice  for  those  in  the  later  stages  of 
adoption. 

Early  majority  Those  who  interact  frequently  with  peers  but  who  may  not  hold  positions  of  opinion  leaders 

within  a  system.  These  individuals  are  an  important  link  between  early  and  late  adopters.  They 
deliberate  before  adopting  an  innovation  but  seldom  take  the  lead  in  adoption. 


Late  majority  Individuals  who  are  typically  skeptical,  who  may  require  some  convincing  or  pressure  to  adopt 
an  innovation.  This  may  come  in  the  form  of  peer  pressure  or  economic  necessity.  Most  of 
the  uncertainty  about  an  innovation's  potential  effectiveness  must  be  resolved  before  these 
individuals  adopt  the  program. 


Laggards  Those  who  possess  no  opinion  leadership  and  are  slow  to  adopt  an  innovation.  They  are 

resistant  to  change  and  suspicious  of  innovations  that  depart  from  traditional  or  past  ways  of 
doing  things.  They  typically  will  adopt  an  innovation  only  if  it  becomes  mainstream  or  a  new 
tradition. 
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The  importance  of  peers  is  also  evident  at  an  organizational  level,  where  social  pressure 
between  organizations  can  promote  adoption.  If  one  organization  or  installation  is  implement¬ 
ing  a  program  or  innovation  with  research  support  and  receiving  accolades,  funding,  or  atten¬ 
tion  for  doing  so,  there  is  social  pressure  for  another  organization  to  adopt  the  model  (Yuan  et 
ah,  2010).  Thus,  coordination  between  peer  organizations  can  help  with  the  diffusion  process 
as  organization  leaders  and  providers  share  empirical  findings  with  others.  Discussions  with 
senior  leaders  in  these  organizations  can  also  facilitate  this  social  pressure. 

Formal  Peer  Involvement 

The  process  of  diffusion  can  follow  a  hierarchical  structure  initiated  by  senior  leaders — 
discussed  in  connection  with  the  fourth  component,  leadership  buy-in  and  support  from  key 
stakeholders — and  disseminated  through  formal  channels  (e.g.,  training).  At  the  beginning  of 
an  initiative,  centralized  training  is  important  for  standardizing  procedures,  generating  rapid 
implementation,  and  safeguarding  training  quality  and  adherence  (Karlin  et  ah,  2010).  The 
next  step  to  broader-scale  implementation  is  decentralized  training,  in  which  trainees  of 
the  certified  trainers  become  certified  themselves  and  train  others  within  an  organization. 
Thus,  in  coordination  with  leadership  support  (the  fourth  component),  they  can  diffuse  knowl¬ 
edge  of  current  research  practices  with  the  anticipation  that  others  will  share  this  knowledge 
with  subordinates  and  peers  throughout  the  system.  A  hierarchical  structure  of  research  dis¬ 
semination  through  peer  networks  may  facilitate  the  diffusion  process.  It  can  also  help  move 
the  program  toward  decentralized  training.  One  of  the  most  successful  national  suicide  pre¬ 
vention  programs  for  youth.  Sources  of  Strength,  trained  peer  leaders  to  disseminate  materials 
about  suicide  prevention  and  recognize  signs  to  refer  friends  who  may  need  adult  assistance 
with  mental  health  concerns  (Wyman  et  ah,  2010).  Peer  leaders  helped  change  the  culture  on 
campuses  by  changing  students’  perceptions  that  adults  at  the  school  would  be  able  to  help 
them  if  they  were  considering  suicide.  Peer  leaders  also  helped  increase  the  acceptability  of 
seeking  help  from  adults  if  needed. 

Informal  Peer  Involvement 

The  Bass  Forecasting  Model  (Mahajan,  2002)  suggests  that  new  ideas  are  more  likely  to  get 
adopted  when  there  is  face-to-face  contact  with  peers.  That  is,  while  ideas  can  be  spread  through 
mass  media  campaigns,  real  change  occurs  only  when  individuals  can  put  an  identity  to  the 
source  of  the  information,  particularly  when  peer  sources  are  credible,  local,  and  consistent  in 
their  messages  (Perlick  et  ah,  2011).  For  example,  peers  can  play  an  important  role  in  stigma 
reduction,  which  is  an  area  of  major  concern  related  to  military  suicide.  Best-practice  sugges¬ 
tions  for  social  marketing  campaigns  aimed  at  strategic  stigma  change  (Corrigan,  2011)  state 
that  contact  with  peers  is  fundamental  to  public  stigma  change  and  that  peer  contact  needs  to 
be  targeted,  credible,  continuous,  and  local  (e.g.,  face  to  face).  These  factors  can  help  dissemi¬ 
nate  messages  about  mental  illness  with  an  eye  toward  cultural  change.  (See  the  case  study  on 
In  Our  Own  Voice  in  Box  6.9  for  an  illustration  of  this  process.) 

Adoption  generally  also  occurs  through  informal  channels  in  which  peers  interact  and 
discuss  programmatic  changes  with  each  other.  Leaders  and  providers  on  the  ground  may  be 
as  important  as  administrative  leaders  in  this  regard  (Barth  and  Sherlock,  2003),  because  they 
may  better  understand  the  culture  of  personnel  working  with  service  members  and  the  day- 
to-day  responsibilities  that  may  hinder  time  and  effort  needed  to  learn  about  new  research 
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Box  6.9 

Case  Study:  In  Our  Own  Voice  (National  Alliance  on  Mental  Illness) 


Identified  Need 

Public  stigma  around  individuals  with  mental  illness  is  well  known. 

Innovation 

The  National  Alliance  on  Mental  Illness  developed  a  program  in  1995  through  a  grant 
from  the  Lilly  Foundation  to  combat  stigma  related  to  mental  illness.  In  Our  Own  Voice 
is  a  brief  (60-minute)  live  presentation  with  videos  presented  by  two  trained  alliance 
members  with  mental  illness.  The  presentation  focuses  on  educating  the  community 
about  individuals  with  mental  illness  and  how  these  individuals  can  lead  productive 
lives  as  they  recover  and  cope  (NAMI,  2011). 

Research 

Support 

Controlled  and  noncontrolled  evaluation  studies  have  demonstrated  that  the  program 
reduced  stigma  attitudes  and  increased  knowledge  about  mental  illness  among 
undergraduate  and  graduate  students  (Rusch  et  al.,  2008;  Wood  and  Wahl,  2006).  It  has 
also  been  shown  to  be  more  effective  than  clinician-delivered  interventions  in  reducing 
stigma  among  family  caretakers  of  individuals  with  mental  illness  (Perlick  et  al.,  2011). 

Key  Materials 

Free  videos,  trainings,  and  available  and  accessible  presentations  promote  engagement 
among  students,  family  members,  and  community  members. 

Status  of 

Diffusion 

Since  its  inception,  the  program  has  reached  over  275,000  people  and  reports  that  it 
currently  reaches  over  50,000  people  per  year  (NAMI,  2011,  p.  6).  It  is  available  free  of 
charge  and  can  be  delivered  in  a  variety  of  settings,  from  family  homes  to  elementary 
schools  and  university  medical  school  classes. 

Peer  network  supportive  of  adoption:  Utilization  of  peers  during  free  presentations 
to  educate  communities  and  families;  targeted,  credible,  continuous,  and  local  contact 
with  peers 

Evidence  of  an  identified  need:  Stigma  about  mental  illness  is  pervasive 

Key  Components 
of  Diffusion 

Evidence  of  research  quality:  Controlled  research  studies  indicate  that  attitudes  can 
change  after  presentations 

Funding  or  other  institutional  support:  Grant  support  and  donations;  endorsed  by 
SAMFISA's  National  Registry  of  Evidence-Based  Programs  and  Practices 

Dissemination  materials:  Free  of  charge,  available  and  accessible,  brief  (one  hour) 

approaches.  Discussions  between  individuals  at  all  levels  in  the  hierarchy  can  help  inform  the 
adoption  process  and  troubleshoot  problems  that  may  hinder  successful  adoption. 

Box  6.9  presents  one  case  study  that  illustrates  this  eighth  component:  In  Our  Own 
Voice,  a  program  developed  by  the  National  Alliance  on  Mental  Illness. 

9.  Dissemination  Materials 

Dissemination  materials  can  be  used  to  educate  providers,  line  leaders,  and  those  on  the  ground 
about  current  and  innovative  research  hndings  that  can  help  inform  their  practices.  These 
materials  can  also  assist  with  efforts  to  diffuse  research  into  practice.  Websites,  brochures, 
pamphlets,  and  marketing  campaigns  can  help  translate  “jargon-y”  academic  research  hndings 
into  layman’s  terms  for  individuals  who  may  not  typically  read  empirical  articles  in  journals. 
Providers  on  the  ground  are  open  and  willing  to  learn  about  hndings  from  research  studies  if 
they  are  presented  in  a  less  esoteric  manner  (Cook,  Biyanova,  and  Coyne,  2009;  Stewart,  Stir- 
man,  and  Chambless,  2012).  Successful  campaigns  have  developed  practical  implementation 
tools  (e.g.,  toolkits,  newsletters,  success  stories)  and  support  (e.g.,  webinars,  conference  calls. 
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trainings)  to  assist  providers  in  implementing  new  practices  (Yuan  et  al.,  2010).  These  tools 
help  address  what  Rogers  (2003)  refers  to  as  “simplicity”:  the  ease  of  use  or  the  perception  that 
a  new  practice  is  easy  to  understand  and  implement. 

Toolkits 

Dissemination  materials  can  include  established  toolkits  to  assist  those  interested  in  imple¬ 
menting  a  new  program  at  their  site.  Several  examples  of  successful  toolkits  are  available  and 
can  help  guide  the  successful  adoption  of  an  innovation  or  evaluation  of  an  already  established 
program.  These  toolkits  are  designed  for  researchers  seeking  to  diffuse  an  innovation,  as  well 
as  organizations  seeking  to  adopt  one;  examples  are  provided  in  Table  6.3. 

Media 

It  is  also  important  to  highlight  programs’  media  exposure  that  can  be  considered  “dissemi¬ 
nation  material.”  This  is  particularly  relevant  for  military  suicide  research,  for  which  there  is 
some  research  support  indicating  that  portrayals  of  suicide  in  news  reports,  film,  web  content, 
and  interactive  training  materials  can  increase  the  risk  of  suicide  contagion  (Hoge  and  Castro, 
2012;  Pirkis  and  Nordentoft,  2011).  By  extension,  it  is  critically  important  that  media  por¬ 
trayals  of  innovative  programs  be  carefully  planned.  Because  the  recent  exposure  of  service 
member  deaths  by  suicide  has  prompted  more  research  and  attention  to  this  highly  important 
area,  it  is  essential  to  recognize  that  many  programs  will  receive  media  coverage  that  may  not 
be  empirically  based.  That  is,  a  new  program  may  appear  attractive  because  it  attempts  to  solve 
a  difficult  problem;  however,  until  there  is  quality  research  supporting  the  innovation,  caution 
must  be  taken  prior  to  adoption.  On  the  flip  side,  media  exposure  of  an  empirically  supported 


Table  6.3 

Examples  of  Toolkits  to  Promote  Dissemination  of  Research  Findings 


Toolkits  for  Researchers  Toolkits  for  Organizations 


AHRQ  has  developed  a  research  dissemination  plan 
tool  for  researchers  to  use  when  designing  research 
studies  (Carpenter  et  al.,  2005).  The  tool  begins 
with  a  question  about  what  the  final  dissemination 
product  will  look  like,  who  is  intended  to  benefit, 
and  how  to  help  targeted  organizations  or 
providers  implement  the  practice.  It  includes  open- 
ended  questions  to  help  researchers  make  a  plan 
for  dissemination  in  the  early  stages  of  research 
design. 


Getting  To  Outcomes^'^:  Promoting  Accountabiiity 
Through  Methods  and  Tools  for  Planning, 
Implementation,  and  Evaluation  is  a  toolkit  to  assist 
established  programs  in  evaluating  their  goals 
and  outcomes  (Chinman,  Imm,  and  Wandersman, 
2004).  Such  a  toolkit  may  be  particularly  important 
when  programs  are  ingrained  into  organizations 
but  lack  empirical  support.  The  toolkit  contains 
worksheets  and  materials  to  guide  the  evaluation 
of  outcomes  by  (1)  recognizing  needs,  goals,  and 
desired  outcomes;  (2)  identifying  the  best-practice 
approaches  to  meet  these  goals;  (3)  assessing  the 
program's  fit  and  the  organizational  capacity 
needed  for  implementation;  and  (4)  conducting 
initial  and  continuous  evaluation. 


The  national  dissemination  plan  developed  by  the  VA's 
Mental  Health  Quality  Enhancement  Research  Initiative 
(Smith  et  al.,  2008)  is  designed  to  disseminate  and  spread 
information  about  evidence-based  collaborative  care  for 
depression  in  primary  care  settings.  The  plan  outlines 
several  approaches  to  effectively  disseminate  collaborative 
care  for  depression  in  the  VA,  such  as  collecting  data, 
documenting  activities,  evaluating  goals  and  flexibility, 
identifying  barriers  and  facilitators  to  the  process  of 
adoption  and  unintended  consequences,  and  soliciting 
perspectives  from  stakeholders  and  providers  to  prepare 
for  dissemination. 

AHRQ's  Preventing  Falls  In  Hospitals:  A  Toolkit  for 
Improving  Quality  of  Care  is  a  toolkit  for  preventing 
falls  in  the  hospital  (AHRQ,  2013;  this  toolkit  is  described 
in  more  detail  in  Box  6.10).  It  provides  many  important 
recommendations  for  translating  research  into  practice 
(Hempel  et  al.,  2012). 
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program  can  help  promote  the  importance  of  evidence-hased  care  among  the  general  public. 
Training  for  program  representatives  on  how  to  speak  to  the  media  about  new  findings  can 
help  the  general  public,  policymakers,  and  others  outside  academic  research  settings  better 
understand  the  state  of  the  research  field  in  a  particular  area  and  why  research-informed  pro¬ 
grams  are  the  best  method  for  addressing  a  problem. 

Technology 

As  technology  becomes  more  advanced,  research  findings  are  becoming  more  accessible  to 
nonacademics.  Many  journal  articles  are  freely  available  online,  as  are  many  empirically  vali¬ 
dated  training  manuals  and  assessment  measures.  Computer-based  training  programs  can  be 
made  accessible  to  providers,  as  can  briefings,  podcasts,  or  video  announcements.  It  is  no 
longer  a  requirement  to  attend  meetings  or  presentations  in  person,  because  more  and  more 
professional  conferences  stream  research  talks  online. 

CPGs  (discussed  in  connection  with  the  seventh  component,  provision  of  incentives  or 
development  of  policies)  can  also  incorporate  new  technology  to  help  with  dissemination.  An 
example  of  a  computer-delivered  CPG  is  outlined  by  Weingardt  and  Villafranca  (2005).  The 
program  follows  the  structure  recommended  by  the  VA/DoD  Clinical  Practice  Guideline  for 
Management  of  Substance  Use  Disorders  (VA  and  DoD,  2009)  to  train  practitioners  in  treat¬ 
ing  alcohol-withdrawal  syndrome  using  a  decision  tree.  The  course  follows  a  case  example  and 
instructs  viewers  in  an  interactive  and  engaging  manner  how  to  provide  adequate  care  for  the 
patient.  The  training  is  brief  and  interactive,  and  it  contains  just  enough  information  “to  get 
the  job  done.”  Nursing  staff  rated  the  program  as  easy  to  use,  useful,  and  appropriate  for  their 
needs. 

In  sum,  dissemination  materials  can  assist  with  the  diffusion  of  innovations.  Success¬ 
ful  programs  utilize  websites,  brochures,  pamphlets,  and  marketing  campaigns,  among  other 
approaches.  Gareful  attention  should  be  paid  to  the  media  portrayal  of  new  practices  developed 
to  address  highly  publicized  issues.  Toolkits  are  available  to  assist  with  dissemination  planning. 

Box  6.10  presents  a  case  study  that  illustrates  this  ninth  component:  AHRQ’s  Preventing 
Falls  in  Hospitals  Toolkit. 

10.  Expectation  of  a  Cultural  Shift 

Rogers  (2003)  refers  to  “compatibility,”  or  how  well  a  new  practice  fits  with  the  potential 
adopter’s  current  experience,  values,  and  goals.  Individuals  exist  within  a  social  system  consist¬ 
ing  of  other  individuals,  groups,  organizations,  and  subsystems,  with  a  common  goal  of  solving 
a  particular  problem  with  the  innovation.  These  individuals  may  have  different  goals,  expecta¬ 
tions,  values,  or  beliefs  related  to  an  innovation,  which  can  delay  adoption. 

Adopters  on  the  ground  are  more  likely  to  adopt  a  new  program  if  it  matches  their  current 
experiences,  values,  and  goals,  and  it  will  be  important  to  demonstrate  this  fit.  Research,  pro¬ 
grams,  and  campaigns  designed  with  an  adopting  organization’s  goals  in  mind  may  be  better 
poised  for  successful  adoption  (Yuan  et  ah,  2010). 

Nonetheless,  it  is  important  to  understand  that  the  natural  process  of  adoption  takes 
time.  Innovations  are  adopted  according  to  a  time  dimension  informed  by  (1)  the  process  by 
which  an  innovation  passes  from  first  knowledge/awareness  to  actual  adoption,  (2)  the  rela¬ 
tive  time  frame  of  the  adoption  (e.g.,  early  or  late  adoption,  as  addressed  in  our  discussion  of 
the  eighth  component),  and  (3)  the  rate  of  adoption  and  number  of  members  in  a  system  who 
adopt  the  innovation  within  a  given  period  (Rogers,  2003). 
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Box  6.10 

Case  Study:  Preventing  Falls  in  Hospitals  Toolkit 


Identified  Need 

Between  700,000  and  1  million  patients  fall  in  hospitals  each  year,  leading  to  injuries 
and  increased  health  care  costs.  However,  many  hospitals  do  not  have  standard 
practices  in  place  for  fall  prevention. 

Innovation 

In  a  joint  effort  by  the  RAND  Corporation,  Boston  University's  School  of  Public 

Health,  and  the  ECRI  Institute,  researchers  developed  an  online  toolkit  for  hospitals  to 
develop,  implement,  and  sustain  a  fall  prevention  program  (AHRQ,  2013).  The  toolkit's 
recommendations  were  generated  from  a  systematic  review  of  the  available  literature 
and,  where  lacking  or  limited,  expert  opinion  about  best  practices  for  hospital  fall 
prevention. 

Research 

Support 

The  toolkit  is  based  on  a  systematic  literature  review  and  expert  opinion  consensus; 
however,  it  was  released  only  recently  and  evidence  supporting  its  utility  is  not  yet 
available. 

Key  Materials 

The  toolkit  contains  a  bibliography  and  citations  throughout  that  direct  the  reader  to 
literature  supporting  each  of  the  recommendations  (Hempel  et  al.,  2012).  Six  questions 
guide  the  consumer  through  a  series  of  recommendations  and  step-by-step  processes 
for  fall  prevention  programs:  (1)  Are  you  ready  for  this  change?  (2)  How  will  you 
manage  change?  (3)  Which  fall  prevention  practices  do  you  want  to  use?  (4)  How  do 
you  implement  best  practices  in  your  organization?  (5)  How  do  you  measure  fall  rates 
and  fall  prevention  practices?  and  (6)  How  do  you  sustain  an  effective  fall  prevention 
program?  The  toolkit  was  released  only  recently;  thus,  evidence  supporting  its  utility  in 
facilitating  the  implementation  of  programs  has  not  yet  been  evaluated. 

Status  of 

Diffusion 

The  toolkit  is  offered  online  and  free  of  charge.  Provider  training  will  help  evaluate  the 
program. 

Key  Components 
of  Diffusion 

Dissemination  materials:  Development  of  an  online  toolkit  for  creating  programs; 
discussion  of  several  other  key  components  of  diffusion  to  overcome  difficulties  and 
assist  with  program  implementation 

Evidence  of  an  identified  need:  A  program  was  needed  to  prevent  accidental, 
preventable  falls  in  hospitals;  demonstrated  a  need  for  the  program  to  leaders, 
stakeholders,  and  staff;  continued  evaluation  and  modification  if  necessary 

Collaboration  with  credible  sponsors;  Collaboration  among  researchers,  universities, 
and  hospitals 

Peer  network  supportive  of  adoption:  Implementation  team;  peer  support  and 
advocacy  of  the  program;  encouraged  continued  enthusiasm  and  support  for  program 

Provision  of  incentives  or  development  of  policies:  Provider  training  and  incorporation 
of  feedback  to  improve  program 

Expectation  of  a  cultural  shift:  Understanding  of  cultural  change  process;  balanced  with 
other  priorities  for  the  population 

One  should  anticipate  resistance  to  cultural  changes  when  providers  and  adopters  on 
the  ground  are  expected  to  implement  a  new  program:  Hahits  of  providers  are  hard  to  break. 
While  it  is  recommended  that  practitioners  modify  their  old  beliefs  that  certain  approaches 
are  the  only  ones  that  work  (Carlton,  2012),  this  is  particularly  difhcult  because  providers  base 
many  of  their  decisions  on  previous  experience  with  patients  rather  than  on  research,  especially 
in  the  absence  of  research  evidence  to  the  contrary  (Dattilio,  Edwards,  and  Fishman,  2010; 
Stewart  and  Chambless,  2007;  Stewart,  Stirman,  and  Chambless,  2012).  If  providers  are  more 


86  Developing  a  Research  Strategy  for  Suicide  Prevention  in  the  Department  of  Defense 


open  to  the  possibility  that  something  else  may  work  better — in  addition  to  or  opposed  to  their 
standard  approach — research  findings  may  inform  practical  work  more  clearly. 

It  is  also  important  to  recognize  that  no  approach — whether  evidence-based  or  not — will 
work  all  the  time  with  every  person  or  in  every  situation.  However,  evidence-based  practices 
have  been  given  a  level  of  scientific  rigor  not  accorded  to  anecdotal  experiences  with  patients. 
There  is  a  traditional  belief  that  clinicians  resist  adapting  beyond  their  training;  however, 
research  indicates  that  providers  are  relatively  open  to  trying  new  evidence-based  approaches 
(Dattilio,  Edwards,  and  Fishman,  2010;  Stewart,  Stirman,  and  Chambless,  2012).  Still,  when 
programs  appeal  to  a  variety  of  theoretical  orientations  or  backgrounds  and  fit  an  identified 
need,  an  innovation  may  be  particularly  welcomed  (e.g.,  Swenson,  2000). 

In  addition,  a  key  principle  of  diffusion  is  reinvention  (Charters  and  Pellegrin,  1973; 
Rogers,  2003),  and  organizations  should  recognize  that  innovations  evolve  over  time  to  meet 
the  needs  of  a  population.  Challenges  are  expected  when  attempting  to  adopt  a  research  prac¬ 
tice,  and  programs  can  prepare  for  those  challenges  ahead  of  time.  Part  of  reinvention  is  involv¬ 
ing  key  players  in  the  diffusion  process  and  incorporating  feedback  to  improve  or  modify  the 
innovation.  By  making  adopters  users  in  the  process  of  redevelopment,  the  innovation  is  more 
likely  to  be  adopted,  accepted,  and  transmitted  among  peers.  For  example,  videogame  compa¬ 
nies  typically  receive  feedback  from  players  on  the  game’s  design;  the  process  can  lead  to  more 
investment  in  the  innovation  and  a  sense  of  ownership,  which,  in  turn,  may  translate  to  suc¬ 
cessful  diffusion. 

The  cultural  change  accompanying  new  program  adoption  will  likely  vary  greatly  depend¬ 
ing  on  the  topic  area,  organization,  individual  personalities,  and  timing  of  the  implementation. 
Following  the  other  nine  components  of  effective  research  diffusion  will  help  overcome  this 
barrier  to  change,  but  it  should  still  be  anticipated  and  planned  for  accordingly.  For  example, 
understanding  that  there  will  be  a  late  majority  and  laggards  during  the  adoption  process 
(Rogers,  2003)  can  help  leadership  and  stakeholders  encourage  participation  in  the  new  pro¬ 
gram  by  providing  incentives  and  peer  support  throughout  the  transition  process.  In  addition, 
providing  training  and  demonstrating  a  need  to  those  on  the  ground  may  also  assist  with  expe¬ 
diting  the  adoption  process. 

New  practices  can  not  only  affect  provider  culture,  but  they  can  also  alter  the  values  and 
beliefs  of  the  organization  (Bradley  et  ah,  2004).  This  can  involve  viewing  program  recipients 
in  a  different  way  and  treating  them  in  a  different  manner.  It  could  also  include  recruiting 
volunteers.  Staff  members  may  have  to  think  about  their  roles  differently  and  work  as  a  team 
across  disciplines.  Again,  patience  will  be  crucial  during  this  time.  Providing  training  and 
continued  support  to  staff  (e.g.,  through  peer  supervision  and  on-site  resources)  have  been 
successful  approaches  in  facilitating  the  continued  adoption  of  new  programs  among  resistant 
providers  (e.g..  Harms  and  Garrard,  1998;  Karlin  et  ah,  2010;  Swenson,  2000). 

Box  6.11  presents  one  case  study  that  illustrates  this  tenth  component:  the  100,000  Lives 
Campaign. 


Conclusion 

In  this  chapter,  we  outlined  ten  components  for  successfully  adopting  research  programs  in 
practice.  The  first  three  components  we  identified  focus  on  demonstrating  that  the  program  is 
needed,  has  the  potential  to  be  successful  at  a  particular  site,  and  is  rooted  in  scientific  evi- 
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Box  6.11 

Case  Study:  The  100,000  Lives  Campaign 


Identified  Need 

Deaths  in  hospitals  can  be  prevented  by  improvements  in  safety  and  the  effectiveness 
of  health  care  approaches. 

Innovation 

The  100,000  Lives  Campaign  is  a  national  initiative  aimed  at  preventing  100,000  hospital 
deaths  (Berwick  et  al.,  2006).  It  utilizes  a  series  of  evidence-based  practices  to  improve 
health  care  provision  in  more  than  3,000  hospitals  nationwide.  The  recommendations 
by  the  original  100,000  Lives  Campaign  were  already  in  practice  (through  not 
standardized)  and  were  promoted  by  federal  or  joint  commission  initiatives  at  adoption, 
facilitating  the  seamless  adoption  of  these  strategies  into  practice.  The  practices 
selected  were  supported  by  empirical  evidence,  which  increased  their  credibility  among 
providers  and  senior  leadership.  The  campaigns  set  up  voluntary  field  offices  to  provide 
support  and  measures  of  quality  improvement  at  the  hospital  sites.  The  campaign  also 
collected  data  on  performance  from  a  majority  of  the  sites. 

Research 

Support 

The  program  has  documented  122,300  lives  saved  (Wachter  and  Pronovost,  2006),  and 

93  percent  of  staff  in  participating  hospitals  are  aware  of  interventions,  with  58  percent 
of  improvements  in  care  attributable  to  campaign  (Sinkowitz-Cochran  et  al.,  2012). 

Key  Materials 

The  offices  provided  implementation  tools  and  support  to  participating  sites.  For 
example,  they  hosted  conference  calls  with  sites  in  rural  settings  with  more  specialized 
needs. 

Status  of 

Diffusion 

The  follow-up  to  this  initiative  was  the  5  Million  Lives  Campaign,  which  added  an 
additional  series  of  evidence-based  practices  and  enrolled  an  additional  1,000  hospitals 
nationally. 

Key  Components 
of  Diffusion 

Expectation  of  a  cultural  shift:  Helped  overcome  this  barrier  with  easy-to-adopt 
practices  and  little  system  redesign;  field  offices  for  support  and  measurement  of 
quality  improvement 

Evidence  of  an  identified  need:  Target  to  prevent  hospital  deaths;  continued  program 
performance  evaluation 

Leadership  buy-in  and  support  from  key  stakeholders;  Gained  support  of  hospitals' 
boards  of  directors,  which  helped  roll  out  the  program  and  garner  support  from 
hospital  staff 

Funding  or  other  institutional  support:  Endorsement  by  federal,  state,  and  national 
organizations 

Provision  of  incentives  or  development  of  policies:  Practices  utilized  by  the  campaign 
were  observable  and  able  to  be  pilot-tested  via  a  trial-and-error  approach. 

Peer  network  supportive  of  adoption:  "Social  pressure"  for  hospitals  to  take  part 
(organization-level  peer  support) 

dence.  Successful  diffusion  begins  with  (1)  establishing  evidence  of  an  identified  need.  This  can 
be  accomplished  by  collecting  data  to  demonstrate  to  organizational  leaders  and  adopters  on 
the  ground  that  the  innovation  has  a  relative  advantage  over  an  existing  program.  The  next  two 
components  relate  to  examining  the  supporting  evidence  for  a  new  program.  Ensuring  that 
the  program  has  (2)  evidence  of  research  quality  and  (3)  evidence  of  real-world  effectiveness  can 
help  inform  leadership  decisions  about  which  programs  to  adopt.  There  are  also  methods  to 
continually  evaluate  established  programs,  and  there  is  a  need  to  continually  examine  whether 
a  new  program  is  meeting  an  organization’s  needs. 
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After  these  first  essential  components,  components  related  to  having  internal  leadership 
and  financial  support  for  a  program  are  warranted.  These  components  include  (4)  leadership 
buy-in  and  support  from  key  stakeholders,  which  can  ensure  that  programs  are  championed 
and  commissioned  hy  individuals  in  a  position  to  execute  a  change  within  an  organization. 
Having  strong  leadership  support  is  essential  to  the  other  nine  components  because  adoption 
is  a  lengthy  process  that  can  he  met  with  resistance.  Most  programs  cannot  exist  without  some 
form  of  (5)  funding  or  other  institutional  support  for  the  program’s  start-up,  hiring  staff,  and  the 
infrastructure  necessary  to  keep  a  program  running  on  a  day-to-day  basis. 

The  remaining  components  deal  primarily  with  the  process  of  garnering  support  and 
disseminating  the  new  program  on  the  ground  level  (i.e.,  with  those  individuals  involved  in 
actual  implementation  of  the  program).  This  can  begin  with  (6)  collaboration  with  credible 
sponsors,  in  which  research  programs  can  be  designed  with  the  target  audiences  involved  at  the 
development  stage.  This  also  includes  collaborations  across  organizations  and,  within  orga¬ 
nizations,  across  departments  and  disciplines  to  promote  innovation.  When  implementing  a 
new  program  into  a  system  that  contains  individuals  who  may  be  complacent  with  the  current 
approach  or  who  are  now  expected  to  perform  additional  or  different  duties,  the  (7)  provi¬ 
sion  of  incentives  or  development  of  policies  can  be  helpful  in  promoting  the  widespread  diffu¬ 
sion  of  the  program.  Incentives  can  come  in  many  forms,  such  as  money,  praise,  or  educa¬ 
tional  opportunities.  It  is  also  important  that  the  new  program  fit  within  the  structure  of  the 
organization — that  it  is  easily  adopted  and  straightforward.  Having  clear  guidelines  and  train¬ 
ings  for  adopters  is  important  here.  The  role  of  (8)  peer  networks  supportive  of  adoption  also 
cannot  be  overestimated,  because  individuals  learn  about  new  practices  from  their  peers. 
Having  peers  champion  a  new  idea  and  support  each  other  in  the  adoption  process  is  a  key 
component  of  diffusion.  Providing  adopters  on  the  ground  with  (9)  dissemination  materials  can 
facilitate  the  process  of  diffusion  by  marketing  and  promoting  ideas.  Step-by-step  toolkits  or 
guidelines  can  assist  in  this  effort.  Finally,  when  a  new  program  is  adopted,  there  should  be 
(10)  the  expectation  of  a  cultural  shift  t\\2X  takes  time  and  will  require  the  continued  support  of 
new  program  adopters.  Awareness  of  and  preparation  for  resistance  and  delays  can  help  lead¬ 
ers  be  flexible  in  meeting  the  needs  of  their  organization  while  supporting  those  charged  with 
implementing  the  new  program. 


CHAPTER  SEVEN 


Recommendations  for  a  Research  Strategy 


DoD  is  one  of  the  largest  funders  of  research  related  to  suicide  prevention  in  the  United  States. 
DoD  currently  funds  62  studies,  accounting  for  close  to  $200  million.  However,  current  fund¬ 
ing  priorities  do  not  reflect  the  department’s  research  needs.  Research  funding  is  overwhelm¬ 
ingly  allocated  to  enhance  prevention  strategies  that  already  are  considered  effective;  those 
considered  most  important,  most  appropriate  for  the  military  context,  and  in  areas  in  which 
the  field  has  the  most  to  learn  receive  relatively  little.  Furthermore,  when  studies  are  completed 
and  new  evidence  emerges  about  a  promising  practice,  DoD,  like  other  organizations,  suf¬ 
fers  from  a  research-to-practice  gap.  Strategies  are  needed  to  ensure  that  the  most  promising 
results,  from  studies  funded  hy  DoD  and  outside  DoD,  can  he  thoughtfully  integrated  into  the 
department’s  operating  procedures  so  that  state-of-the-art  techniques  are  in  place  to  prevent 
the  further  loss  of  lives  to  suicide. 

In  light  of  these  issues,  we  offer  a  series  of  ten  recommendations.  The  first  is  an  over¬ 
arching  recommendation,  and  the  remaining  nine  are  spread  across  three  general  categories: 
(1)  areas  in  which  DoD  should  prioritize  research  funding;  (2)  processes  DoD  should  adopt 
or  enhance  to  more  efficiently  allocate  research  funding;  and  (3)  processes  DoD  should 
adopt  or  enhance  to  ensure  that  evidence-supported  suicide  prevention  strategies  are  integrated 
into  current  operations. 


Overarching  Recommendation 

1.  Leadership  is  needed  to  provide  strategic  guidance  for  impiementing  a  unified  research 
strategy. 

There  are  at  least  six  funding  streams  in  DoD  for  research  on  suicide  prevention,  each  managed 
hy  a  different  office,  and  there  are  likely  more  defense  entities  that  have  funded  this  research  in 
the  past  and  plan  to  do  so  in  the  future.  The  success  of  a  strategic  and  unified  DoD  research 
strategy  will  depend  on  those  who  oversee  these  funding  streams  to  collahorate  with  each  other 
and  work  synergistically,  ensuring  that  the  right  research  is  being  funded  and  that  research 
findings  are  being  disseminated  to  appropriate  audiences.  Leadership  is  also  needed  to  enact 
and  be  accountable  for  the  remaining  nine  recommendations  presented  here,  from  creating, 
housing,  and  managing  a  directory  of  ongoing  research  studies  to  reevaluating  research  priori¬ 
ties  in  light  of  new  and  novel  findings.  Ensuring  that  these  objectives  are  accomplished  will 
require  that  an  individual  or  a  single,  specific  office  is  designated  as  the  lead  entity  to  direct  and 
oversee  the  suicide  prevention  research  portfolio. 
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In  choosing  where  these  responsibilities  are  housed,  the  Secretary  of  Defense  should  con¬ 
sider  which  entities  are  in  the  best  position  to  supply  the  necessary  leadership  and  oversight. 
The  responsibilities  of  this  individual  or  office  would  include  providing  senior  oversight  of  the 
suicide  prevention  research  portfolio,  ensuring  accountability  across  organizations  and  entities 
involved  with  funding  and  implementing  the  research,  ensuring  strategic  coordination  among 
all  stakeholders  within  DoD,  and  making  decisions  about  which  research  funding  should  be 
prioritized  and  what  interventions  should  be  disseminated  into  practice.  The  designated  indi¬ 
vidual  or  office  will  need  to  have  the  appropriate  level  of  authority  to  ensure  coordination  across 
service  components  and  defense  agencies,  but  also  at  a  level  high  enough  within  DoD  to  com¬ 
mand  the  attention  of  these  entities.  This  may  be  a  role  for  the  Office  of  the  Under  Secretary  of 
Defense  for  Personnel  and  Readiness,  which  houses  the  Suicide  Prevention  Office  and  the  Per¬ 
sonnel  Risk  Reduction  Office  (under  the  supervision  of  the  Military  Deputy)  and  the  Office 
of  the  Assistant  Secretary  of  Defense  for  Health  Affairs.  However,  it  should  be  noted  that  the 
Army  is  currently  the  lead  funding  agency  for  research  on  suicide  prevention,  with  U.S.  Army 
Medical  Research  and  Materiel  Command  managing  82  percent  of  DoD’s  funding  for  suicide 
prevention  in  addition  to  the  significant  investment  for  Army  STARRS.  Thus,  a  case  may  be 
made  for  the  Army  to  have  a  significant  leadership  or  authority  role  as  an  executive  agent  for 
the  department.  Nonetheless,  having  a  single  point  of  authority  and  responsibility  will  serve 
to  foster  the  further  integration  and  unification  of  DoD  suicide  prevention  research  activities. 


Priorities  for  Research  Funding 

2.  Eliciting  the  opinions  of  relevant  stakeholders  can  inform  the  development  of  DoD's 
research  priorities. 

We  had  experts  rank  suicide  prevention  strategies  across  five  domains,  including  cultural 
acceptability  and  future  learning  potential;  we  also  calculated  a  suicides-prevented  index  for 
each  strategy.  By  these  metrics,  two  strategies  stand  out.  The  first  is  ensuring  that  non— health 
professionals  (i.e.,  noncommissioned  officers,  chaplains)  who  come  in  contact  with  suicidal  individu¬ 
als  are  trained  to  identify,  care  for,  and  refer  persons  at  risk,  which  currently  accounts  for  5  per¬ 
cent  of  spending  on  suicide  prevention  research  in  DoD.  This  strategy’s  high  level  of  cultural 
acceptability  is  likely  because  such  gatekeeper  training  is  the  primary  suicide  prevention  strat¬ 
egy  used  in  the  Army,  Marine  Corps,  and  Navy.  The  second  is  implementing  population-based 
programs  that  reduce  suicide  risk  factors  and  build  resilience  (i.e.,  risk  reduction),  which  currently 
accounts  for  10  percent  of  DoD’s  suicide  prevention  research  spending.  While  prediction, 
screening,  and  biological  interventions  are  all  deemed  as  having  low  cultural  acceptability  and 
low  benefit-cost  index  values,  they  have  moderate  levels  of  future  learning  potential.  This  sug¬ 
gests  that  these  strategies  should  not  be  abandoned. 

We  believe  that  the  domains  we  used  to  rank  research  priorities  are  relevant,  but  they  are 
certainly  not  exhaustive.  Criteria  other  than  cost,  cultural  acceptability,  and  future  learning 
potential  may  be  important  to  consider  when  prioritizing  suicide  prevention  research.  This 
does  not  make  the  results  of  the  current  study  futile;  the  method  that  we  employed  is  repli¬ 
cable,  so  DoD  could  conceivably  elicit  feedback  on  alternative  domains  and  rank  research  goals 
across  new  criteria. 

That  said,  as  important  as  it  is  to  choose  the  relevant  domains,  it  is  equally  important  to 
interpret  the  rankings  that  result.  For  example,  a  strategy  that  is  not  deemed  culturally  accept- 
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able  according  to  our  criteria  or  that  currently  has  a  low  suicides-prevented-per-cost  value  could 
translate  into  significant  long-term  returns.  The  implication  is  that  portfolio  managers  should 
ensure  that  strategies  with  both  short-  and  long-term  expected  returns  have  adequate  research 
attention.  Similarly,  our  criteria  for  future  learning  potential  suggest  that  if  two  assets  have  the 
same  benefit-cost  index,  the  portfolio  manager  should  invest  more  in  the  one  with  the  higher 
learning  value,  because  there  is  value  to  resolving  uncertainty.  The  striking  conclusion  here — 
to  actually  invest  more  in  uncertain  assets — should  not  be  viewed  as  a  contradiction  to  tradi¬ 
tional  portfolio  theory.  Traditional  portfolio  theory  assumes  that  investment  actions  cannot 
influence  the  amount  of  uncertainty  and  thus  encourages  diversification  to  decrease  the  level  of 
uncertainty.  With  a  research  portfolio,  the  portfolio  manager  is  equipped  with  the  additional 
option  of  decreasing  uncertainty  by  actively  learning  from  the  investment,  thus  achieving  the 
same  goal. 

3.  Research  investment  is  needed  to  prioritize  strategies  with  iow  suicides-prevented-per- 
cost  vaiues;  poiicy  changes  are  needed  to  make  aiready  high  suicides-prevented-per-cost 
strategies  more  cuituraiiy  acceptabie. 

As  mentioned  earlier,  strategies  that  we  estimate  as  having  low  suicides-prevented-per-cost 
values  do  so  either  because  they  are  costly  to  implement  or  because  current  evidence  does  not 
suggest  that  they  are  effective.  However,  as  we  have  mentioned  previously,  continued  research 
investment  in  these  areas  is  useful  and  necessary  to  create  new  approaches  or  to  modify  exist¬ 
ing  approaches  that  can  make  them  both  less  costly  and  more  effective. 

However,  there  are  strategies  that  are  already  thought  to  be  effective  but  that  are  not 
deemed  culturally  acceptable  in  the  military  context,  in  particular  (1)  encouraging  service 
members  and  their  families  to  be  knowledgeable  about  and  proactively  seek  treatment  {stigma 
reductiori),  (2)  reducing  service  members’  access  to  the  means  by  which  they  might  take  their 
own  lives  {reduced  access)',  and  (3)  improving  psychosocial  interventions  used  by  clinicians 
(e.g.,  psychiatrists,  psychologists,  social  workers)  to  identify  and  treat  those  at  risk  for  suicide 
{psychosocial  interventions).  It  is  through  policy  change  and  awareness  campaigns  that  the  cul¬ 
ture  of  the  military  can  change  to  ensure  that  these  promising  suicide  prevention  strategies  can 
achieve  their  full  potential  in  preventing  suicide  among  service  members.  Research  and  evalu¬ 
ation  studies  should  accompany  any  such  changes  so  that  their  value  or  impact  on  suicide  rates 
can  be  investigated. 

4.  Funding  agencies  in  DoD  shouid  make  a  proactive  effort  to  fund  effectiveness  research, 
in  which  interventions  that  prior  research  (funded  by  DoD  or  another  entity)  has  deemed 
efficacious  are  evaiuated  for  their  effectiveness  in  the  miiitary  context. 

There  are  a  number  of  research  studies  related  to  suicide  prevention  that  have  some  degree  of 
efficacy.  However,  how  these  approaches  may  work  in  DoD  is  unknown.  It  would  be  appropri¬ 
ate  for  DoD  to  examine  existing  approaches  with  evidence  of  efficacy  and  sponsor  effective¬ 
ness  trials  to  determine  how  the  approach  might  work  in  the  military  and  what  adaptations 
may  be  needed  to  maximize  benefits.  Effectiveness  studies  test  whether  research  programs  can 
be  translated  successfully  into  real-world  settings  like  DoD  and  whether  the  programs  have 
the  components  necessary  to  achieve  such  diffusion.  Ensuring  that  such  studies  are  part  of  a 
research  strategy  will  help  facilitate  the  diffusion  of  research  into  practice. 

One  example  of  a  study  that  could  benefit  from  effectiveness  research  in  DoD  is  the 
“caring  letters”  trial,  originally  conducted  in  1976.  In  that  study,  a  randomly  selected  group 
of  suicide  attempters  who  had  refused  treatment  were  sent  outreach  letters  from  program  staff 
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over  a  four-year  period;  after  the  first  two  years  of  the  study,  those  sent  these  letters  were  less 
likely  to  die  hy  suicide  than  a  control  group  (Motto  and  Bostrom,  2001).  In  2011,  MOMRP 
funded  David  Luxton’s  study  of  the  feasibility  of  applying  this  approach  to  patients  seen  at  a 
military  treatment  facility.  (See  Appendix  A  for  details.)  This  feasibility  study  found  that  the 
approach  could  be  successfully  implemented  in  that  setting;  however,  the  authors  reiterated 
the  need  for  an  RCT  to  evaluate  whether  those  who  receive  these  messages  are  ultimately  less 
likely  to  die  by  suicide. 


Processes  to  Enhance  and  More  Efficiently  Allocate  Research  Funding 

5.  DoD  should  have  a  central  repository  to  track  the  research  it  is  funding  on  suicide 
prevention. 

The  discussion  of  current  research  in  Chapter  Two  provides  a  “snapshot”  of  suicide  prevention 
research  at  a  single  point  in  time.  New  studies  are  being  funded  routinely,  and  the  information 
in  Chapter  Two  is  already  dated.  While  a  comprehensive  database  of  funding  levels  and  trends 
would  be  valuable,  it  would  need  to  be  updated  somewhat  regularly  with  up-to-date  informa¬ 
tion  (i.e.,  annually).  In  creating  the  catalog  of  current  research,  the  RAND  study  team  found 
identifying  DHHS-  and  AFSP-funded  research  to  be  relatively  easy,  as  both  agencies  have 
updated,  publicly  accessible  databases  on  their  studies.  In  contrast,  analyzing  current  trends 
in  suicide  prevention  research  is  more  complicated  because  there  is  no  central  repository  for 
DoD  suicide  prevention  research  and  because  online  databases  are  lacking  in  the  VA.  We  rec¬ 
ommend  that  a  central  repository  be  developed  and  automatically  updated  over  time,  so  that 
it  is  more  efficient  to  take  a  snapshot  of  what  kind  of  suicide  prevention  research  is  being  con¬ 
ducted.  To  the  extent  that  DoD  agencies  are  pursuing  such  databases,  efforts  to  keep  them  up 
to  date  should  be  sustained,  as  should  efforts  to  make  them  compatible  with  other  federal  data¬ 
bases  (e.g.,  in  DHHS).  Not  only  would  such  a  repository  benefit  DoD,  but  it  would  also  ensure 
that  those  responsible  for  funding  research  know  what  is  already  being  funded  and  provide 
stakeholders  outside  DoD,  such  as  in  DHHS  and  AFSP,  with  ready  access  to  this  information. 

6.  The  designated  leadership  agency  in  DoD  (per  recommendation  1)  should  continually 
reevaluate  its  research  priorities  in  light  of  new  research  findings,  new  policies,  and  the 
adoption  of  new  suicide  prevention  strategies. 

We  ranked  aspirational  goals  according  to  five  constructs:  importance,  effectiveness,  cultural 
acceptability,  cost,  and  future  learning  potential.  The  ranking  of  aspirational  goals  against 
these  domains  is  dynamic  and  may  change  over  time.  Specifically,  new  research  findings  may 
shift  how  strategies  rank  with  respect  to  efficacy  and  future  learning  potential;  policy  changes 
may  shift  the  cultural  acceptability  of  certain  policies;  and  the  implementation  of  new  preven¬ 
tion  strategies  in  DoD,  such  as  prediction  modeling,  may  heighten  the  importance  of  research 
in  that  area.  For  example,  enhancing  the  confidentiality  afforded  to  service  members  who  seek 
behavioral  health  care  may  enhance  the  cultural  acceptability  of  strategies  like  help-seeking, 
quality  care,  and  both  biological  and  psychosocial  interventions.  It  is  important  to  reevaluate 
the  rankings  over  time,  especially  if  these  values  will  be  used  to  parameterize  the  model  we 
present  in  Chapter  Four. 
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Processes  to  Close  the  Research-to-Practice  Gap 

7.  DoD  should  encourage  both  formal  and  informal  collaboration  across  DoD  entities 
responsible  for  funding  and  implementing  suicide  prevention  programs  and  strategies. 

Suicide  prevention  strategies  cut  across  a  number  of  organizations  and  agencies  within  DoD. 
There  is  division  in  terms  of  occupation.  For  example,  chaplains  and  noncommissioned  officers 
play  a  critical  role  in  gatekeeper  training,  and  psychiatrists,  psychologists,  social  workers,  and 
other  behavioral  health  care  providers  are  critically  important  for  ensuring  quality  care  and 
continuity  of  care,  as  well  as  ultimately  treating  patients  with  psychological  or  biological  inter¬ 
ventions.  In  addition,  there  is  further  subdivision  by  service.  The  leadership,  peer  networks, 
and  organizational  structure  of  each  of  these  groups  varies,  and  the  process  of  implementing 
research  into  practice  will  inevitably  differ  for  each.  As  such,  it  is  critical  that  both  formal  and 
informal  relationships  are  established  or  enhanced  to  promote  the  diffusion  of  research  into 
practice. 

As  an  example,  consider  the  MSRC  study,  “Usability  and  Utility  of  a  Virtual  Hope 
Box  (VHB)  for  Reducing  Suicidal  Ideation”  (see  Gutierrez  and  Bush,  2012).  Hope  boxes  are 
common  strategies  used  by  behavioral  health  care  providers  and  are  physical  representations 
of  the  patient’s  reasons  for  living.  Thus,  patients  can  refer  to  these  items  during  times  of  hope¬ 
lessness  and  despair.  Bush  is  creating  and  testing  a  virtual  hope  box  application  that  patients 
can  download  on  their  smartphones.  To  diffuse  this  technology  into  routine  clinical  care, 
behavioral  health  care  providers  who  treat  military  personnel  must  know  that  the  application 
exists  and  trust  that  it  works;  patients  would  also  have  to  trust  the  security  of  the  content  deliv¬ 
ered.  This  would  mean  establishing  relationships  both  with  leaders  of  each  service’s  behavioral 
health  teams  and,  potentially,  with  individual  behavioral  health  care  providers  at  different 
installations  or  military  treatment  facilities.  Similarly,  commands  would  have  to  be  engaged, 
know  that  the  mobile  device  is  a  component  of  the  service  member’s  treatment,  and  make  the 
appropriate  allowances.  Collaborations  between  these  entities  early  on  in  the  process  of  diffu¬ 
sion  will  be  critical  for  the  success  of  any  innovation. 

8.  Agencies  that  fund  suicide  prevention  research  and  those  responsible  for  implementing 
suicide  prevention  programs  should  keep  abreast  of  new  research,  bearing  in  mind 

the  quality  of  different  studies.  Efficiencies  may  be  gained  by  creating  a  centralized 
clearinghouse  for  this  purpose,  perhaps  capitalizing  on  existing  sources. 

There  are  more  than  200  ongoing  studies  related  to  suicide  prevention,  of  which  fewer  than 
half  are  being  conducted  in  DoD.  It  is  important  to  monitor  the  results  of  these  studies  while 
considering  the  quality  of  each.  It  is  through  monitoring  these  studies’  results  that  research  can 
inform  the  suicide  prevention  strategies  that  DoD  chooses  to  adopt.  DoD  may  consider  estab¬ 
lishing  its  own  process  for  monitoring  study  results  or  may  rely  on  resources  that  already  exist. 
In  particular,  as  referenced  in  Chapter  Five,  SAMHSA’s  National  Registry  of  Evidence-Based 
Programs  and  Practices  (SAMHSA,  2014)  and  the  Suicide  Prevention  Resource  Center’s  Best 
Practices  Registry  (Suicide  Prevention  Resource  Center,  undated)  offer  descriptions  of  differ¬ 
ent  strategies,  how  they  have  been  evaluated,  and  the  quality  of  these  evaluations.  DoD  could 
use  this  information  to  ensure  that  the  programs  it  advocates  and  implements  reflect  the  state 
of  the  art. 
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9.  Agencies  and  organizations  within  DoD  shouid  be  encouraged  to  adopt  evidence- 
based  technoiogies.  Such  encouragement  may  inciude  funding,  materiais,  and  technicai 
assistance. 

Services,  agencies,  and  organizations  within  DoD  may  decide  to  try  to  implement  a  research- 
informed  strategy  for  suicide  prevention;  alternatively,  DoD  may  encourage  the  services,  agen¬ 
cies,  or  organizations  to  implement  a  specific  research-informed  strategy.  Either  way,  it  should 
foster  such  adoption  and  can  do  so  hy  providing  real  support  to  the  agencies  responsible  for 
implementation.  This  can  mean  providing  funding  to  cover  some  of  the  costs  associated  with 
implementing  the  strategy,  dissemination  materials  to  encourage  the  adoption  of  the  new 
approach,  or  even  technical  assistance  to  help  organizations  implement  and  adapt  the  approach 
in  their  unique  settings. 

10.  Both  ieadership  buy-in  and  peer  engagement  are  key  in  promoting  new  technoiogies. 

Leadership  involvement  is  critically  important  in  disseminating  research  into  practice.  In  fact, 
one  of  the  evidence-hased  practices  for  suicide  prevention,  listed  in  both  SAMHSA’s  National 
Registry  of  Evidence-Based  Programs  and  Practices  and  the  Suicide  Prevention  Resource  Cen¬ 
ter’s  Best  Practices  Registry,  is  the  Air  Eorce’s  Suicide  Prevention  Program.  The  first  tenet  of 
the  program  is  “leadership  involvement,”  which  involves  the  release  of  messages  related  to  sui¬ 
cide  prevention  every  three  to  six  months  from  the  Chief  of  Staff  of  the  Air  Eorce.  As  stated 
in  Air  Eorce  Pamphlet  44-160:  “It  is  encouraging  that  Air  Eorce  leaders,  even  at  the  Squadron 
level,  continue  to  see  suicide  prevention  as  important.  Sending  regular  messages  over  the  past 
four  years  from  the  Chief  of  Staff  of  the  Air  Eorce  is  one  reason  for  this  sustained  interest” 
(U.S.  Air  Eorce,  2001,  p.  11). 

However,  perhaps  as  important  as  leadership  involvement  is  the  involvement  of  peers. 
To  effectively  involve  peers  in  disseminating  new  research-based  interventions,  DoD  would 
benefit  from  a  strategy  that  identifies  the  group  responsible  for  administering  the  intervention, 
discerns  how  leadership  will  be  involved  in  promoting  the  intervention,  and  engages  peers  in 
dissemination  efforts.  This  may  occur  formally;  for  example,  a  selected  number  of  personnel 
are  taught  about  the  intervention  and  then  train  their  peers  in  how  to  apply  it.  However,  it 
may  also  be  strategic  for  peer  involvement  to  occur  informally  so  that  personnel  can  observe 
their  peers  using  a  new  approach  and  see  that  it  works.  Either  strategy  will  most  likely  involve 
partnering  with  services,  organizations,  and  leaders  to  identify  innovators  and  early  adopters 
who  will  be  most  receptive  to  using  a  new  prevention  strategy. 


Conclusion 

The  final  recommendation  of  the  congressionally  mandated  Task  Eorce  on  the  Prevention  of 
Suicide  by  Members  of  the  Armed  Eorces  was  for  DoD  to  “create  a  unified,  strategic,  and  com¬ 
prehensive  DoD  plan  for  research  in  military  suicide  prevention  ensuring  that  DoD’s  military 
suicide  prevention  research  portfolio  is  thoughtfully  planned  to  cover  topics  in  prevention, 
intervention,  and  postvention”  (U.S.  Department  of  Defense  Task  Eorce  on  the  Prevention 
of  Suicide  by  Members  of  the  Armed  Eorces,  2010,  p.  ES-18).  In  light  of  a  surprising  lack  of 
research  on  effective  suicide  prevention  strategies  to  date,  task  force  members  understood  that 
if  DoD  is  to  make  progress  in  reducing  suicide,  research  must  be  a  fundamental  component 
of  its  strategy.  The  data  presented  in  this  report  and  the  recommendations  we  draw  from  our 


Recommendations  for  a  Research  Strategy  95 


analysis  represent  the  ways  in  which  DoD  can  continue  to  support  research  that  can  ultimately 
he  incorporated  into  operating  procedures  and,  ideally,  reduce  the  burden  of  suicide  on  mili¬ 
tary  personnel. 


APPENDIX  A 


Ongoing  Studies  of  Relevance  to  Suicide  Prevention  Among 
Military  Personnel 


Tables  A.1-A.5  present  key  information  about  ongoing  studies  with  relevance  to  suicide  pre¬ 
vention  among  military  personnel.  The  studies  are  categorized  by  funder:  Army  STARRS, 
DoD,  VA,  DHHS,  or  AFSR 
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Table  A.1 

The  Army  Study  to  Assess  Risk  and  Resilience  in  Servicemembers  (Army  STARRS) 


Start-End  Funding  (S) 


Researching 

Principai  Investigator(s)  Organizations  Participants  NAASP  Category  MOMRP  Category 


08/09- 

09/14 


65,000,000 

Contributors: 
DoD:  50,000,000 
NIMH:  15,000,000 


Robert  J.  Ursano,  M.D.,  of  the 
Uniformed  Services  University  of 
the  Health  Sciences 

Murray  Stein,  M.D.,  M.P.H.,  of 
the  University  of  California,  San 
Diego. 

Steven  G.  Heeringa,  Ph.D., 
University  of  Michigan 

Ronald  C.  Kessler,  Ph.D.,  Harvard 
Medical  School 

NIMH  scientists  include 
Lisa  J.  Coipe,  Ph.D.,  M.P.H. 
Michael  Schoenbaum,  Ph.D. 


NIMH 

Harvard  Medical  School 
University  of  Michigan 
Columbia  University 
University  of  California, 
San  Diego,  Medical 
School 

U.S.  Army 
Henry  M.  Jackson 
Foundation  for  the 
Advancement  of 
Military  Medicine 


Army 

Active  duty 

National 

Guard/reserves 


NIMH  and  Army  oversight  and 
implementation  leadership 
include 

Scott  Ludtke,  GS-15,  Acting  Army 
Executive  Director,  Army  STARRS 


Kevin  Quinn,  Ph.D.,  NIMH  Study 
Director,  Army  STARRS 


Enhanced  continuity  of  care 
Provider  and  gatekeeper 
training 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Risk  and  protective  factor 
interactions 

Stigma  reduction 
Population-based  risk 
reduction/resilience-building 
Prevention  of  reattempts 
Prediction  of  imminent  risk 

Improved  biological 
interventions 

Reduction  in  access  to  lethal 
means 

Population-based  screening 


Prevention,  education,  and 
training 

Early  screening/intervention 

Assessment 

Treatment 

Recovery  and  return  to  duty 
Postvention 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 


James  Churchill,  Ph.D.,  NIMH 
Program  Officer,  Army  STARRS 


98  Developing  a  Research  Strategy  for  Suicide  Prevention  in  the  Department  of  Defense 


Table  A. 2 

DoD-Funded  Suicide  Prevention  Studies 


Funding 

Dept. 

Start- 

End 

Funding 

(S) 

Titie 

Principal 

Investigator(s) 

Researching 

Organization 

Participants 

NAASP  Category 

MOMRP  Category 

AFRL 

09/12- 

09/13 

325,000 

Resiliency  and 
Personnei  Weli- 
Being  and  Suicide 
Risk 

Kerry  L.  Knox, 
Ph.D. 

University 
of  Rochester 
Medical  Center 

Air  Force 

Risk  and  protective  factor 
interactions 

Population-based  risk 
reduction/resilience-building 

Prevention,  education,  and 
training 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

AFRL 

09/11- 

08/13 

1,000,000 

New  Technologies, 
Social  Networks, 
Mental  Health,  and 
Suicide  Prevention 

Laura  Miller, 
Ph.D. 

RAND 

Corporation 

Air  Force 

Psychosocial  interventions 
for  those  at  risk 

Risk  and  protective  factor 
interactions 

Population-based  screening 

Prevention,  education,  and 
training 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

AFRL 

2007- 

ongoing 

9,000,000 

Pre-Service 
Emotional 
Adjustment  and 
In-Service  Suicide 
Attempts 

Kristin 

Schneider,  Ph.D. 

Defense 
Advanced 
Research  Projects 
Agency 

Air  Force 

Risk  and  protective  factor 
interactions 

Population-based  risk 
reduction/resilience-building 

Postvention 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

CDMRP 

2011- 

2012 

200,000 

Stigma  Data 

Analysis 

Crosby  Hipes, 
Ph.D. 

New  York 
University 

Army 

Stigma  reduction 

Prevention,  education,  and 
training 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

CDMRP 

2008- 

2012 

271,547 

Deployment,  PTSD 
Symptoms,  and 
Comorbid  Mental 
Health  Conditions 
in  the  Active 

Force  and  Reserve 
Components 

Laurel  Hourani, 
Ph.D.,  M.P.H 

Research 

Triangle  Institute 

Active-duty 

National 

Guard/ 

reserves 

Psychosocial  interventions 
for  those  at  risk 

Risk  and  protective  factor 
interactions 

Prevention,  education,  and 
training 

Early  screening/intervention 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 
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Table  A. 2 — Continued 


Funding 

Dept. 

Start- 

End 

Funding 

(S) 

Title 

Principal 

Investigator(s) 

Researching 

Organization 

Participants 

NAASP  Category 

MOMRP  Category 

CDMRP 

09/08- 

01/11 

352,167 

Mental  Health  and 
Resilience:  Soldiers' 
Perceptions  about 
Psychotherapy, 
Medication,  and 
Barriers  to  Care  in 
the  United  States 
Military 

Steven 

Southwick,  M.D. 

Yale  University 

Active  duty 

Affordable,  accessible,  and 
effective  care 

Risk  and  protective  factor 
interactions 

Stigma  reduction 

Prevention,  education,  and 
training 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

CDMRP 

09/10- 

04/13 

354,287 

Transcranial 
Magnetic 
Stimulation:  Effect 
of  TMS  vs.  Sham  on 
Suicidality 

Mark  S.  George, 
M.D. 

Medical 

University  of 
South  Carolina 

Veterans 

Improved  biological 
interventions 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

CDMRP 

2010- 

2013 

375,675 

Do  Undetected  or 
Untreated  Sleep 
Disorders  Predict 
PTSD  Psychotherapy 
Outcomes? 

Patricia  L 

Haynes,  Ph.D. 

Biomedical 
Research 
Foundation 
of  Southern 
Arizona 

Veterans 

Risk  and  protective  factor 
interactions 

Prediction  of  imminent  risk 

Early  screening/intervention 
Epidemiology  and/or 
basic  science/neurological 
mechanisms 

CDMRP 

04/09- 

01/13 

519,608 

Combat,  Sexual 
Assault,  and  Post- 
Traumatic  Stress  in 
OIF/OEF  Military 
Women 

Anne  G.  Sadler, 
R.N.,  Ph.D. 

VA  Medical 
Center,  Iowa  City 

Active  duty 

Population-based  screening 

Prevention,  education,  and 
training 

CDMRP 

08/09- 

09/12 

636,732 

PTSD  Trajectory, 
Comorbidity, 
and  Utilization 
of  Mental  Health 
Services  Among 
National  Guard 
Soldiers 

Robert  J. 

Ursano,  M.D. 

Henry  M. 

Jackson 

Foundation 

National 

Guard/ 

reserves 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Risk  and  protective  factor 
interactions 

Prevention,  education,  and 
training 

Assessment 

Treatment 

Stigma  reduction 
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Table  A. 2 — Continued 


Funding 

Dept. 


CDMRP 


CDMRP 


CDMRP 


CDMRP 


CDMRP 


Start-  Funding  Principai  Researching 

End  (S)  Titie  Investigator(s)  Organization 


03/09-  765,029  Reintegration:  Linda  O.  Nichols,  VA  Medical 

06/13  The  Role  of  Ph.D.  Center,  Memphis 

Spouse  Telephone 
Battlemind 


2008- 

2011 


09/08- 

10/11 


2008- 

2011 


2008- 

2009 


780,638 

Dissemination  of 
Evidence-Based 

CBT  Intervention 
Components: 

Online  Self- 
Administered 
Training  for 
Providers 

Treating  Military 
Deployment- 
Related  PTSD 

Stephen  Rao, 
Ph.D. 

New  England 

Research 

Institutes 

Cleveland  Clinic 
Foundation 

835,270 

PTSD  Trajectory, 
Comorbidity, 
and  Utilization 
of  Mental  Health 
Services  Among 
Reserves 

Sandro  Galea, 
M.D.,  M.P.H., 
Dr.PH. 

Columbia 

University 

887,166 

Effectiveness  of 
Cognitive  Exposure 
and  Skills  Group 
Manuallzed 
Treatments  in 
OIF/OEF  Female 
Veterans 

Diane  Castillo, 
Ph.D. 

VA  Medical 

Center, 

Albuquerque 

920,000 

Titrating  Optimal 
Delivery  of 
Mindfulness- 
Based  Training 
Interventions 

Amishi  P.  Jha, 
Ph.D. 

University  of 
Pennsylvania/ 
University  of 
Miami 

Participants 


NAASP  Category 


MOMRP  Category 


Active  duty 

Families 

Provider  and  gatekeeper 
training 

Affordable,  accessible,  and 
effective  care 

Population-based  risk 
reduction/resilience-building 

Prevention,  education,  and 
training 

Assessment 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

Veterans 

Provider  and  gatekeeper 
training 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Assessment 

Treatment 

National 

Guard/ 

reserves 

Affordable,  accessible,  and 
effective  care 

Prevention,  education,  and 
training 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

Veterans 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Treatment 

Active  duty 

Provider  and  gatekeeper 
training 

Affordable,  accessible,  and 
effective  care 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 
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Table  A. 2 — Continued 


Funding 

Dept. 

Start- 

End 

Funding 

(S) 

Title 

Principal 

Investigator(s) 

Researching 

Organization 

CDMRP 

09/12- 

08/15 

1,087,300 

Adaptive  Disclosure: 
A  Combat-Specific 
PTSD  Treatment 

Ariel  Lang,  Ph.D. 

University  of 
California,  San 
Diego 

CDMRP 

2010 

1,152,019 

Facilitating  Soldier 
Receipt  of  Needed 
Mental  Health 
Treatment 

Thomas  W. 

Britt,  Ph.D. 

Clemson 

University 

CDMRP 

03/08- 

03/14 

1,445,213 

Innovative 

Service  Delivery 
for  Secondary 
Prevention  of  PTSD 
in  At-Risk  OIF-OEF 
Service  Men  and 
Women 

Ron  E.  Acierno, 
Ph.D. 

VA  Medical 
Center, 
Charleston, 
South  Carolina 

CDMRP 

2011- 

2015 

1,700,000 

Optimizing 

Delivery  of 
Mindfulness-Based 
Military  Training 
Interventions  in 
Army  Infantry 
Cohorts 

Amishi  P.  Jha, 
Ph.D. 

University  of 
Pennsylvania/ 
University  of 
Miami 

CDMRP 

2007 

1,725,303 

The  Impact 
of  Supported 
Employment 

Versus  Standard 
Vocational 
Rehabilitation  In 
Veterans  With  PTSD 

Dr.  Lori  L.  Davis, 
M.D. 

Tuscaloosa 
Research  and 
Education 
Advancement 

CDMRP 

2007 

2,018,015 

Homecoming  Line: 
Telephone  Support 
for  Veterans 

Craig  H.  Rosen, 
M.D. 

Stanford 

University 

Participants  NAASP  Category  MOMRP  Category 


Marines 


Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 


Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 


Active  duty  Affordable,  accessible,  and  Prevention,  education,  and 
effective  care  training 

Provider  and  gatekeeper 
training 


Active  duty 


Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 


Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 


Army 


Provider  and  gatekeeper 
training 


Prevention,  education,  and 
training 

Treatment 


Veterans 


Population-based  risk 
reduction/resilience-building 


Prevention,  education,  and 
training 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 


Veterans  Affordable,  accessible,  and  Treatment 
effective  care 

Psychosocial  interventions 
for  those  at  risk 
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Table  A. 2 — Continued 


Funding 

Dept. 

Start- 

End 

Funding 

(S) 

Titie 

Principal 

Investigator(s) 

Researching 

Organization 

Participants 

NAASP  Category 

MOMRP  Category 

CDMRP 

2009- 

2011 

2,070,446 

Telemental  Health 
and  Cognitive 
Processing  Therapy 
for  Female  Combat 
Veterans  with 
Military-Related 
PTSD 

Leslie  Morland, 
Psy.D. 

VA  Research 
and  Education 
Corporation  of 
the  Pacific 

Veterans 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

CDMRP 

2007 

2,143,061 

Telemedicine  for 
Improved  Delivery 
of  Psychosocial 
Treatments  for 
Post-Traumatic 

Stress  Disorder 

Steven  R.  Thorp, 
Ph.D. 

Veterans 

Medical  Research 
Foundation  of 

San  Diego 

Veterans 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

CDMRP 

2010 

2,778,806 

PHIT  for  Duty,  a 
Personal  Health 
Intervention  Tool 
for  Psychological 
Health  and 
Traumatic  Brain 
Injury 

Paul  N. 

Kizakevich,  M.S. 

Research 

Triangle 

Institute 

Active  duty 

Veterans 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Treatment 

CDMRP 

08/07- 

09/13 

2,849,739 

Effectiveness  of 
Telerehabilitation 
for  OIF/OEF 
Returnees  with 
Combat-Related 
Trauma 

Kris 

Siddharthan, 

Ph.D. 

VA  Medical 
Center,  Orlando 

Active  duty 

Veterans 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

CDMRP 

2007- 

2014 

3,770,255 

A  Multisite, 
Randomized 

Clinical  Trial  of 
Virtual  Reality  and 
Prolonged  Exposure 
Therapy  for  Active 
Duty  Soldiers  with 
PTSD 

Gregory  Gahm, 
Ph.D. 

National  Center 
for  Telehealth 
and  Technology 

Active  duty 

Enhanced  continuity  of  care 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Risk  and  protective  factor 
interactions 

Population-based  risk 
reduction/resilience-building 

Prevention,  education,  and 
training 

Assessment 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 
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Table  A. 2 — Continued 


Funding 

Dept. 

Start- 

End 

Funding 

(S) 

Titie 

Principal 

Investigator(s) 

Researching 

Organization 

Participants 

NAASP  Category 

MOMRP  Category 

CDMRP 

2007- 

2011 

4,204,171 

Telemental  Health 
and  Cognitive 
Processing  Therapy 
for  Rural  Combat 
Veterans  with  PTSD 

Leslie  A. 
Morland,  Psy.D. 

VA  Research 
and  Education 
Corporation  of 
the  Pacific 

Veterans 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Treatment 

CDMRP 

2010 

5,130,000 

Trauma 
Management 
Therapy  for  OEF 
and  OIF  Combat 
Veterans 

Deborah  C. 
Beidel,  Ph.D. 

University  of 
Central  Florida 

Veterans 

Enhanced  continuity  of  care 

Provider  and  gatekeeper 
training 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Risk  and  protective  factor 
interactions 

Prevention,  education,  and 
training 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

CDMRP 

2012 

9,886,992 

Reduction  in 

Suicide  Risk:  A 
Double-Blind, 
Placebo-Controlled 
Trial  of  Omega-3 
Fatty  Acid 
Supplementation 
among  Military 
Veterans 

Bernadette  P. 
Marriott,  Ph.D. 

Medical 
University  of 
South  Carolina 

Veterans 

Risk  and  protective  factor 
interactions 

Stigma  reduction 

Prevention  of  reattempts 
Improved  biological 
interventions 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

DCoE 

09/07- 

08/11 

65,786 

Building 
Neurocognitive 
Resilience  with 
Attention  Training 
in  a  Military  Cohort 

Amishi  P.  Jha, 
Ph.D. 

University  of 
Pennsylvania 

Army 

Population-based  risk 
reduction/resilience-building 

Prevention,  education,  and 
training 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

DCoE 

08/10- 

12/13 

648,802 

Review  of  Suicide 
Prevention:  Toolkit 
for  Evaluating 
Suicide  Prevention 
Programs  and 
Postvention 
Programs  and  Policy 

Rajeev 

Ramchand, 

Ph.D. 

RAND 

Corporation 

Army 

Air  Force 

Navy 

Marines 

Enhanced  continuity  of  care 
Affordable,  accessible,  and 
effective  care 

Risk  and  protective  factor 
interactions 

Prevention,  education,  and 
training 

Postvention 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 
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Funding 

Dept. 

Start- 

End 

Funding 

(S) 

Title 

Principal 

Investigator(s) 

Researching 

Organization 

Participants 

NAASP  Category 

MOMRP  Category 

DCoE 

09/12- 

03/14 

1,000,000 

Stigma  Reduction 
Efforts  in  the 
Department  of 
Defense 

Joie  Acosta, 

Ph.D. 

RAND 

Corporation 

Active  duty 

Stigma  reduction 

Prevention,  education,  and 
training 

DCoE 

09/12- 

03/14 

1,000,000 

Availability 
and  Efficacy  of 
Military-Culture 
Appropriate 
Psychological 

Health  Treatment 
for  Geographically 
Distant  Service 
Members  and  Their 
Families 

Ryan  Brown, 
Ph.D.,  Grant 
Marshall,  Ph.D. 

RAND 

Corporation 

Active  duty 

National 

Guard/ 

reserves 

Veterans 

Families 

Affordable,  accessible,  and 
effective  care 

Prevention,  education,  and 
training 

DCoE 

08/10- 

09/14 

4,095,985 

Innovative  Practices 
for  Supporting 
Psychological 

Health  and  TBI 

Carrie  Farmer, 
Ph.D.,  Deborah 
Scharf,  Ph.D., 
Robin  Weinick, 
Ph.D. 

RAND 

Corporation 

Active  duty 

Enhanced  continuity  of  care 

Affordable,  accessible,  and 
effective  care 

Risk  and  protective  factor 
interactions 

Prevention,  education,  and 
training 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

Dept,  of 
the  Navy 

05/11- 

05/14 

6,989,000 

Marine  Resiliency 
Study 

Many 

University  of 
California,  San 
Diego 

Naval  Health 
Research  Center 

U.S.  Department 
of  the  Navy 

Marines 

Enhanced  continuity  of  care 

Provider  and  gatekeeper 
training 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Prevention,  education,  and 
training 

Early  screening/intervention 

Assessment 

Treatment 

Epidemiology  and/or 
basic  science/neurological 

Risk  and  protective  factor  mechanisms 
interactions 


Stigma  reduction 

Population-based  risk 
reduction/resilience-building 
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Table  A. 2 — Continued 


Funding 

Dept. 


DoD 


DoD 


Start-  Funding  Principai  Researching 

End  (S)  Titie  Investigator(s)  Organization 


2001-  Ongoing  The  Millennium 

2022  funding  Cohort  Study 


Nancy  F.  Crum- 
Cianflone,  M.D., 
M.P.H. 


Army 
Navy 
Air  Force 
VA 

Many  academic 
institutions 


2007- 

2013 


35,989,697  The  STRONG  STAR 
Multidisciplinary 
PTSD  Research 
Consortium 


Lt  Col  Alan  L. 
Peterson,  Ph.D 
A.B.P.P. 

COL  Stacey 
Young- 
McCaughan, 
Ph.D.,  R.N. 


University  of 
Texas,  Austin 

Health  Science 
Center  at  San 
Antonio 


Participants  NAASP  Category  MOMRP  Category 


Air  Force 

Army 

Navy 

Marines 

National 

Guard/ 

reserves 

Veterans 


Enhanced  continuity  of  care 
Provider  and  gatekeeper 
training 

Affordable,  accessible,  and 
effective  care 
Psychosocial  interventions 
for  those  at  risk 
Risk  and  protective  factor 
interactions 
Stigma  reduction 
Population-based  risk 
reduction/resilience-building 
Prediction  of  imminent  risk 
Improved  biological 
interventions 

Reduction  in  access  to  lethal 
means 

Population-based  screening 


Prevention,  education,  and 
training 

Early  screening/intervention 

Assessment 

Treatment 

Recovery  and  return  to  duty 
Postvention 
Epidemiology  and/or 
basic  science/neurological 
mechanisms 


Active  duty 
Veterans 


Enhanced  continuity  of  care 
Provider  and  gatekeeper 
training 

Affordable,  accessible,  and 
effective  care 
Psychosocial  interventions 
for  those  at  risk 
Risk  and  protective  factor 
interactions 
Stigma  reduction 
Population-based  risk 
reduction/resilience-building 
Prevention  of  reattempts 
Prediction  of  imminent  risk 
Improved  biological 
interventions 

Reduction  in  access  to  lethal 
means 

Population-based  screening 


Prevention,  education,  and 
training 

Early  screening/intervention 

Assessment 

Treatment 

Recovery  and  return  to  duty 
Postvention 
Epidemiology  and/or 
basic  science/neurological 
mechanisms 
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Funding  Start-  Funding  Principai 

Dept.  End  (S)  Titie  Investigator(s) 


MOMRP  N/A 


Data  Land  Combat  Study  Paui  Y.  Kim, 
missing  of  an  Army  Infantry  M.A. 
Division  2003-2009 


MOMRP 

09/12- 

Data 

Suicide 

09/14 

missing 

Bereavement 
in  Veterans  and 
Military  Families 

Julie  Cerel,  Ph.D. 


MOMRP  08/09- 
09/11 


137,000  Suicide  in  the  MAJ  Owen  T. 

Active  Duty  Army  Hill 
2000-2009 


MOMRP  09/11-  218,000  Posttraumat/c  Stress  Valerie  Stander, 

04/13  Disorder,  Substance  Ph.D. 

Abuse  and  Self 
Harm:  Mediating 
Relationships  with 
Respect  to  Combat 
Stress 


MOMRP  09/11- 
04/13 


282,040  Drug-Related  Toby  Cooper, 

Overdoses  Among  a  Pharm.D., 
Military  Population  B.C.P.S.,  R.Ph. 


MOMRP  09/09-  656,184  Antidepressants  Matthew  Miller, 

09/11  and  the  Risk  of  M.D. 

Self-Harm  and 
Unintentional 
Injury  Among 
Younger  Veterans 


Researching 

Organization 


Walter  Reed 
Army  Institute 
of  Research 


University  of 
Kentucky 


U.S.  Army 
Research 
Institute  of 
Environmental 
Medicine 

Naval  Health 
Research  Center 


Darnell  Army 
Medical  Center 


Harvard 

University 


Participants 


NAASP  Category 


MOMRP  Category 


Army 

Risk  and  protective  factor 
interactions 

Population-based  risk 
reduction/resilience-building 

Prevention,  education,  and 
training 

Early  screening/intervention 
Epidemiology  and/or 
basic  science/neurological 
mechanisms 

Veterans 

Families 

Psychosocial  interventions 
for  those  at  risk 

Postvention 

Army 

Population-based  screening 

Prevention,  education,  and 
training 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

Navy 

Marines 

Provider  and  gatekeeper 
training 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Assessment 

Treatment 

Active  duty 

National 

Guard/ 

reserves 

Veterans 

Risk  and  protective  factor 
interactions 

Population-based  risk 
reduction/resilience-building 

Prevention,  education,  and 
training 

Postvention 

Veterans 

Risk  and  protective  factor 
interactions 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 
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Table  A. 2 — Continued 


Funding 

Dept. 

Start- 

End 

Funding 

(S) 

Titie 

Principal 

Investigator(s) 

Researching 

Organization 

MOMRP 

09/09- 

09/12 

753,159 

Optimizing 
Screening  and  Risk 
Assessment  for 
Suicide  Risk  in  the 
U.S.  Military 

Thomas  Joiner, 
Ph.D. 

Florida  State 
University 

MOMRP 

08/09- 

09/13 

1,173,408 

Blister  Packaging 
Medication  to 
Increase  Treatment 
Adherence  and 
Clinical  Response: 
Impact  on  Suicide- 
Related  Morbidity 
and  Mortality 

Peter  Gutierrez, 
Ph.D. 

Denver  VA 
Medical  Center 

MOMRP 

09/09- 

09/12 

1,279,912 

High-Risk 

Suicidal  Behavior 
in  Veterans — 
Assessment 
of  Predictors 
and  Efficacy  of 
Dialectical  Behavior 
Therapy  (DBT) 

Marianne 
Goodman,  M.D. 

VA  Medical 
Center,  New 
York 

MOMRP 

07/11- 

08/13 

1,800,000 

Development 
and  Validation  of 
a  Theory-Based 
Screening  Process 
for  Suicide  Risk 

Steven  Vannoy, 
Ph.D.,  M.P.H. 

University  of 
Washington 

MOMRP 

09/09- 

09/12 

1,961,003 

The  Association 
Between  Suicide 
and  OEF/OIF 
Deployment  History 

Mark  Reger, 
Ph.D. 

National  Center 
for  Telehealth 
and  Technology 

MOMRP 

09/09- 

09/12 

1,967,035 

Brief  Cognitive 
Behavioral  Therapy 
for  Military 
Populations 

David  Rudd, 
Ph.D. 

Texas  Tech 
University 

Participants  NAASP  Category  MOMRP  Category 


Army 

Affordable,  accessible,  and 
effective  care 

Risk  and  protective  factor 
interactions 

Early  screening/intervention 

Assessment 

Veterans 

Civilians 

Psychosocial  interventions 
for  those  at  risk 

Risk  and  protective  factor 
interactions 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

Veterans 

Psychosocial  interventions 
for  those  at  risk 

Prediction  of  imminent  risk 

Early  screening/intervention 

Treatment 

Army 

Risk  and  protective  factor 
interactions 

Prediction  of  imminent  risk 

Assessment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

Active  duty 

National 

Guard/ 

reserves 

Veterans 

Risk  and  protective  factor 
interactions 

Prevention,  education,  and 
training 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

Active  duty 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Treatment 
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Funding 

Dept. 

Start- 

End 

Funding 

($) 

Titie 

Principal 

Investigator(s) 

Researching 

Organization 

Participants 

NAASP  Category 

MOMRP  Category 

MOMRP 

02/09- 

06/13 

2,095,025 

Identification  of  At- 
Risk  Interventions 
for  Pre-Deployment 
Psychophysiologic 
Predictors  of 
Postdeployment 
Mental  Health 
Outcomes 

Jeffrey  M.  Pyne, 
Ph.D. 

Biomedical 

Research 

Foundation, 

Little  Rock, 
Arkansas 

National 

Guard/ 

reserves 

Psychosocial  interventions 
for  those  at  risk 

Risk  and  protective  factor 
interactions 

Prediction  of  imminent  risk 

Early  screening/intervention 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

MOMRP 

11/11- 

02/16 

2,500,000 

Caring  Letters  for 
Military  Suicide 
Prevention:  A 
Randomized 
Controlled  Trial 

David  D  Luxton, 
Ph.D. 

National  Center 
for  Telehealth 
and  Technology 

Active  duty 

National 

Guard/ 

reserves 

Veterans 

Enhanced  continuity  of  care 
Provider  and  gatekeeper 
training 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Prevention,  education,  and 
training 

MOMRP 

08/12- 

08/15 

3,000,000 

Intranasal  Delivery 
of  Biodegradable 
Neuropeptide 
Nanoparticles 

Michael  J. 

Kubek,  Ph.D. 

Indiana 

University  School 
of  Medicine 

Civilians 

Improved  biological 
interventions 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

MOMRP 

04/11- 

08/15 

3,400,000 

A  Randomized 
Clinical  Trial  of 
the  Collaborative 
Assessment  and 
Management 
of  Suicidality  vs. 
Enhanced  Care  as 
Usual  for  Suicidal 
Soldiers 

David  A.  Jobes, 
Ph.D. 

Catholic 
University  of 
America 

Army 

Enhanced  continuity  of  care 

Population-based  risk 
reduction/resilience-building 

Prediction  of  imminent  risk 

Assessment 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

MOMRP- 

Uniformed 

Services 

University 

of  the 

Health 

Sciences 

08/12- 

08/15 

663,741 

Pilot  Trial  of 
Inpatient  Cognitive 
Therapy  for  the 
Prevention  of 
Suicide  in  Military 
Personnel  with 
Acute  Stress 

Disorder  or  PTSD 

Marjan 

Holloway,  Ph.D. 

Uniformed 

Services 

University  of  the 
Health  Sciences 

Veterans 

Psychosocial  interventions 
for  those  at  risk 

Prevention  of  reattempts 

Treatment 
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Table  A. 2 — Continued 


Funding 

Dept. 

Start- 

End 

Funding 

(S) 

Titie 

Principal 

Investigator(s) 

Researching 

Organization 

Participants 

NAASP  Category 

MOMRP  Category 

MOMRP- 

Uniformed 

Services 

University 

of  the 

Health 

Sciences 

09/09- 

09/12 

2,666,717 

A  Brief  Intervention 
to  Reduce  Suicide 
Risk  in  Military 
Service  Members 
and  Veterans — 
Study  2  (SAFEMIL) 

Marjan 

Holloway,  Ph.D. 

Uniformed 

Services 

University  of  the 
Health  Sciences 

Active  duty 

Veterans 

Psychosocial  interventions 
for  those  at  risk 

Treatment 

MOMRP- 

Uniformed 

Services 

University 

of  the 

Health 

Sciences 

08/09- 

09/14 

6,000,000 

Post-Admission 
Cognitive  Therapy 
(PACT)  for  the 
Inpatient  Treatment 
of  Military 

Personnel  with 
Suicidal  Behaviors: 

A  Multi-Site  RCT 

Marjan 

Holloway,  Ph.D. 

Uniformed 

Services 

University  of  the 
Health  Sciences 

Veterans 

Psychosocial  interventions 
for  those  at  risk 

Treatment 

MOMRP- 

MSRC 

N/A 

Data 

missing 

Usability  and  Utility 
of  a  Virtual  Hope 
Box  (VHB)  for 
Reducing  Suicidal 
Ideation 

Nigel  Bush, 

Ph.D. 

National  Center 
for  Telehealth 
and  Technology 

Active  duty 

Veterans 

Affordable,  accessible,  and 
effective  care 

Risk  and  protective  factor 
interactions 

Early  screening/intervention 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

MOMRP- 

MSRC 

09/11- 

10/15 

Data 

missing 

Military  Continuity 
Project 

Katherine  A. 
Comtois,  Ph.D., 
M.P.H. 

University  of 
Washington 

Marines 

Affordable,  accessible,  and 
effective  care 

Stigma  reduction 

Prevention,  education,  and 
training 

MOMRP- 

MSRC 

09/11- 

08/13 

367,588 

Development 
and  Evaluation 
of  a  Brief,  Suicide 
Prevention 
Intervention 
Reducing  Anxiety 
Sensitivity 

Norman 

Schmidt,  Ph.D. 

Florida  State 
University 

Veterans 

Civilians 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Risk  and  protective  factor 
interactions 

Prevention,  education,  and 
training 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

MOMRP- 

MSRC 

11/11- 

09/14 

1,013,904 

Brief  Interventions 
for  Short-Term 

Craig  J.  Bryan, 
Ph.D. 

University  of 
Utah,  National 

Army 

Psychosocial  interventions 
for  those  at  risk 

Treatment 

Suicide  Risk  Center  for 

Reduction  in  Veterans 

Military  Populations 


110  Developing  a  Research  Strategy  for  Suicide  Prevention  in  the  Department  of  Defense 


Table  A. 2 — Continued 


Funding 

Dept. 

Start- 

End 

Funding 

(S) 

Titie 

Principai 

Investigator(s) 

Researching 

Organization 

Participants 

NAASP  Category 

MOMRP  Category 

MOMRP- 

MSRC 

09/09- 

09/12 

1,173,408 

Toward  a  Gold 
Standard  for  Suicide 
Risk  Assessment  for 
Military  Personnel 

Thomas  Joiner, 
Ph.D.,  and  Peter 
Gutierrez,  Ph.D. 

Denver  VA 
Medical  Center 

Air  Force 

Army 

Navy 

Prediction  of  imminent  risk 

Assessment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

MOMRP- 

MSRC 

09/11- 

08/13 

1,182,369 

A  Behavioral  Sleep 
Intervention  for 
the  Prevention 
of  Suicidal 

Behaviors  in 

Military  Veterans: 

A  Randomized 
Controlled  Trial 

Rebecca 

Bernert,  Ph.D. 

Stanford 

University 

Veterans 

Affordable,  accessible,  and 
effective  care 

Risk  and  protective  factor 
interactions 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

MOMRP- 

MSRC 

01/12- 

01/14 

Window  to  Hope: 
Evaluating  a 
Psychological 
Treatment  for 
Hopelessness 

Among  Veterans 
with  Traumatic 

Brain  Injury 

Lisa  Brenner, 
Ph.D. 

VA  Rocky 

Mountain 

Network 

(Veterans 

Integrated 

Service  Network 

19) 

Veterans 

Psychosocial  interventions 
for  those  at  risk 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

TJ 

fD 

1/1 

O 

3 

3 

fD 
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Table  A. 3 

VA-Funded  Suicide  Prevention  Studies 


Start-  Principal  Researching 


End 

Funding  ($) 

Title 

Investigator(s) 

Organization 

Participants 

NAASP  Category 

MOMRP  Category 

04/10- 

04/11 

92,777 

Pain  and  Suicidal 
Behaviors  Among 
Veterans  in  Care  in 
the  VA 

Joseph  Lucien 
Goulet,  Ph.D., 
M.S. 

VA  Connecticut 
Healthcare 
System,  West 
Haven  Campus 

Veterans 

Psychosocial  interventions  for  those 
at  risk 

Risk  and  protective  factor 
interactions 

Assessment 

Epidemiology  and/or  basic  science/ 
neurological  mechanisms 

Prediction  of  imminent  risk 

01/12- 

12/12 

86,950 

Stakeholder 
Perspectives  on 
Improving  Access 
to  VHA's  Suicide 
Prevention  Services 

Monica 

Matthieu,  Ph.D., 
M.S.W. 

St.  Louis  VA 
Medical  Center, 
John  Cochran 
Division 

Veterans 

Enhanced  continuity  of  care 
Population-based  screening 

Assessment 

09/11- 

08/14 

877,257 

Veteran  Interactions 
with  VA  Primary  Care 
Prior  to  Suicide 

Steven  K. 
Dobscha,  M.D. 

Portland  VA 
Medical  Center 

Veterans 

Enhanced  continuity  of  care 
Population-based  risk  reduction/ 
resilience-building 

Population-based  screening 

Prevention,  education,  and  training 

07/05- 

06/12 

756,588 

Suicide  Among 
Veterans:  Using 
the  VA  Depression 
Registry  to  Inform 
Care 

Marcia  T. 
Valenstein,  M.D. 

VA  Ann  Arbor 

Healthcare 

System 

Veterans 

Psychosocial  interventions  for 
those  at  risk 

Risk  and  protective  factor 
interactions 

Assessment 

Epidemiology  and/or  basic  science/ 
neurological  mechanisms 

10/08- 

04/12 

892,700 

Outcomes  and 
Correlates  of  Suicidal 
Ideation  in  OEF/OIF 
Veterans 

Steven  K. 
Dobscha,  M.D. 

Portland  VA 
Medical  Center 

Veterans 

Risk  and  protective  factor 
interactions 

Assessment 

Epidemiology  and/or  basic  science/ 
neurological  mechanisms 

09/09- 

08/10 

95,500 

Implementation  and 
Refinement  of  the 

Lisa  Brenner, 
Ph.D. 

VA  Rocky 
Mountain 

Active  duty 

Veterans 

Population-based  risk  reduction/ 
resilience-building 

Prevention,  education,  and  training 

Suicide  Classification  Network 

System  (Veterans 

Integrated  Service 
Network  19) 
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Start-  Principal  Researching 

End  Funding  ($)  Title  Investigator(s)  Organization  Participants  NAASP  Category 


2011- 

2013 

69,025 

Brief  CBT  for 

Insomnia  in  VA 
Primary  Care  Patients 
With  Depression  and 
Insomnia  to  Reduce 
Suicide  Risk 

Wilfred  R. 
Pigeon,  Ph.D. 

VA  Center  of  Veterans 

Excellence 

for  Suicide 

Prevention 

(Veterans 

Integrated  Service 

Network  2) 

Psychosocial  interventions  for 
those  at  risk 

Risk  and  protective  factor 
interactions 

01/11- 

12/11 

100,000 

Documenting 
Variability  in  Suicide 
Event  Reporting 

Robert  Bossarte, 
Ph.D. 

VA  Center  of  Veterans 

Excellence 

for  Suicide 

Prevention 

(Veterans 

Integrated  Service 

Network  2) 

Population-based  screening 

08/10- 

07/15 

1,098,989 

Assessing  MedicationsEr\c  G.  Smith, 
as  Interventions  to  M.D.,  M.P.H. 

Prevent  Suicide  in  the 

VHA 

Edith  Nourse  Veterans 

Rogers  Memorial 

Veterans 

Hospital,  Bedford, 
Massachusetts 

Psychosocial  interventions  for 
those  at  risk 

Risk  and  protective  factor 
interactions 

08/09- 

07/13 

832,420 

Patient  and  Provider 
Outcomes  of 
E-Learning  Training 
in  Collaborative 
Assessment  and 
Management  of 
Suicidality 

Kathryn  M. 
Magruder, 

Ph.D.,  M.P.H. 

Ralph  H.  Veterans 

Johnson  VA 

Medical  Center, 

Charleston,  South 

Carolina 

Enhanced  continuity  of  care 

Psychosocial  interventions  for  those 
at  risk 

Risk  and  protective  factor 
interactions 

04/09- 

09/10 

387,711 

Management  of 
Suicidal  Veterans 
during  Substance  Use 
Disorder 

Mark  A.  Ilgen, 
Ph.D. 

VA  Ann  Arbor  Veterans 

Healthcare 

System,  Ann 

Arbor,  Ml 

Reduction  in  access  to  lethal 

means 

Psychosocial  interventions  for  those 
at  risk 

Risk  and  protective  factor 
interactions 


MOMRP  Category 

Treatment 

Epidemiology  and/or  basic  science/ 
neurological  mechanisms 


Prevention,  education,  and  training 

Epidemiology  and/or  basic  science/ 
neurological  mechanisms 


Treatment 

Epidemiology  and/or  basic  science/ 
neurological  mechanisms 

Prevention,  education,  and  training 

Epidemiology  and/or  basic  science/ 
neurological  mechanisms 


Prevention,  education,  and  training 
Treatment 
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Table  A. 3 — Continued 


Start- 

End 

Funding  ($) 

Title 

Principal 

Investigator(s) 

Researching 

Organization 

Participants 

NAASP  Category 

MOMRP  Category 

07/05- 

08/09 

756,588 

Suicide  Among 
Veterans:  Using 
the  VA  Depression 
Registry  to  Inform 
Care 

Marcia  T. 
Valenstein,  M.D. 

VA  Ann  Arbor 
Healthcare 

System 

Veterans 

Enhanced  continuity  of  care 

Prediction  of  imminent  risk 

Prevention,  education,  and  training 

Assessment 

04/12- 

03/17 

823,624 

Motivational 
Interviewing  to 
Prevent  Suicide  in 
High  Risk  Veterans 

Peter  Britton, 
Ph.D.,  M.S. 

Syracuse  VA 
Medical  Center 

Veterans 

Prediction  of  imminent  risk 

Assessment 

Epidemiology  and/or  basic  science/ 
neurological  mechanisms 

10/09- 

09/13 

654,318 

Neurobiology  of 
Suicide  Risk  in 
Traumatic  Brain 
Injury  and 

Substance  Abuse 

Deborah 

Yurgelun-Todd, 

Ph.D. 

VA  Rocky  Veterans 

Mountain 

Network 

(Veterans 

Integrated  Service 

Network  19) 

Prediction  of  imminent  risk 

Improved  biological  interventions 

Assessment 

Epidemiology  and/or  basic  science/ 
neurological  mechanisms 

04/08- 

03/12 

601,175 

Serotonin  2C 
Receptor  mRNA 
Editing  in  Suicide 

Stella  Dracheva 
(Plevan),  Ph.D. 

Bronx  VA 

Hospital 

Veterans 

Prediction  of  imminent  risk 

Improved  biological  interventions 

Assessment 

Epidemiology  and/or  basic  science/ 
neurological  mechanisms 
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Table  A. 4 

DHHS-Funded  Suicide  Prevention  Studies 


Funding 

Dept. 

Start- 

End 

Funding  (S) 

Title 

Principal 

Investigator(s) 

Researching 

Organization 

Participants 

NAASP  Category 

MOMRP  Category 

AHRQ 

01/10- 

06/13 

1,476,745 

Health-Related 
Quality  of  Life 
in  Teens  with 
Depression-HRQL 
Teen  Depression 

Frances  Lynch, 
Ph.D. 

Kaiser  Foundation 

Civilians 

Risk  and  protective  factor 
interactions 

Early  screening/intervention 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

AHRQ 

08/10- 

06/12 

299,576 

The  Public  Health  Nicoletta  Tessler, 

Trust  Jackson  Health  M.A. 

System  Proposes  to 

Improve  Patients 

Public  Health  Trust  Civilians 
of  Miami-Dade 

County,  Florida 

Affordable,  accessible,  and 
effective  care 

Population-based  risk 
reduction/resilience-building 

Assessment 

CDC-NCIPC 

09/08- 

08/11 

897,518 

Prospective  Risk  and  Malfred  van 
Protective  Factors  Dulmen,  Ph.D. 
for  Suicide  and 

Co-Occurring  Risk 

Behaviors 

Kent  State 
University 

Civilians 

Risk  and  protective  factor 
interactions 

Assessment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

CDC-NCIPC 

09/10- 

09/15 

396,100 

Links  to 

Enhancing  Teens' 
Connectedness 
(LET'S  CONNECT) 

Cheryl  King, 

Ph.D. 

University  of 
Michigan 

Civilians 

Affordable,  accessible,  and 
effective  care 

Early  screening/intervention 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

CDC-NCIPC 

08/10- 

07/13 

399,930 

Etiology  of  Suicidal  Kenneth  Conner, 
Behavior  During  Ph.D. 

Adolescence  and 

Emerging  Adulthood 

University  of 
Rochester 

Civilians 

Risk  and  protective  factor 
interactions 

Prediction  of  imminent  risk 

Early  screening/intervention 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

National 
Center  for 
Infectious 
Diseases 

09/09- 

08/11 

698,270 

College  Students 
with  Elevated 
Suicide  Risk: 
Enhancing 
Treatment  Linkage 

Cheryl  A.  King, 
Ph.D. 

University  of 
Michigan 

Civilians 

Affordable,  accessible,  and 
effective  care 

Risk  and  protective  factor 
interactions 

Prediction  of  imminent  risk 

Early  screening/intervention 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 
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Table  A. 4 — Continued 


Funding  Start-  Principal  Researching 

Dept.  End  Funding  (S)  Title  Investigator(s)  Organization  Participants  NAASP  Category  MOMRP  Category 


National 
Institute 
on  Alcohol 
Abuse  and 
Alcoholism 

OS/OS- 

OS/IB 

157,021 

Alcohol,  Suicide 
and  HIV  Prevention 
of  Teens  in  Mental 
Health  Treatment 

Christianne  George  Mason 

L.  Esposito-  University 

Smythers,  Ph.D. 

Civilians 

Population-based  risk  Prevention,  education,  and 

reduction/resilience-building  training 

Population-based  screening  Early  screening/intervention 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

National 

Institute 

09/11- 

08/14 

344,918 

Acute  Alcohol  Use 
and  Suicide 

Mark  S.  Kaplan,  Portland  State 

Ph.D.  University 

Veterans 

Psychosocial  interventions 
for  those  at  risk 

Prevention,  education,  and 
training 

on  Alcohol 
Abuse  and 
Alcoholism 

Risk  and  protective  factor  Early  screening/intervention 

interactions  Epidemiology  and/or 

Population-based  risk  basic  science/neurological 

reduction/resilience-building  mechanisms 

National 
Institute 
on  Alcohol 
Abuse  and 
Alcoholism 

04/11- 

03/13 

203,436 

Suicidal  Ideation 
and  Alcohol 
Outcomes  In 
Emerging  Adult 
College  Drinkers 

Vivian  M.  University  of 

Gonzalez,  Ph.D.  Alaska,  Anchorage 

Civilians 

Risk  and  protective  factor 
interactions 

Early  screening/intervention 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

National 
Institute 
on  Alcohol 
Abuse  and 
Alcoholism 

08/11- 

07/13 

146,077 

Novel  Web-Based 
Intervention  for 
Heavy  Drinking  and 
Depressed  College 
Students 

Irene  M.  Geisner,  University  of 

Ph.D.  Washington 

Civilians 

Psychosocial  interventions 
for  those  at  risk 

Treatment 

National 
Institute 
on  Alcohol 
Abuse  and 
Alcoholism 

08/12- 

07/14 

214,906 

The  Acute  Alcohol- 
Suicide  Attempt 
Relation  as  a 
Function  of  Alcohol 
Use  Disorders 

Courtney  L.  University  of 

Bagge,  Ph.D.  Mississippi  Medical 

Center 

Civilians 

Psychosocial  interventions 
for  those  at  risk 

Risk  and  protective  factor 
interactions 

Prediction  of  imminent  risk 

Assessment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

Population-based  screening 
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Table  A. 4 — Continued 


Funding  Start-  Principal  Researching 

Dept.  End  Funding  (S)  Title  Investigator(s)  Organization  Participants  NAASP  Category 


Eunice 

09/04- 

2,049,838  Youth  Violence 

Kennedy 

08/08 

Exposure:  A 

Shriver 

National 

National 
Institute  on 
Child  Health 
and  Human 
Development 

Longitudinal  Study 

Dean  G.  Medical  University  Civilians 

Kilpatrick,  Ph.D.,  of  South  Carolina 

M.D. 


Eunice  09/06- 

Kennedy  08/10 

Shriver 

National 

Institute  on 

Child  Health 

and  Human 

Development 


234,000  School-Based  Richard  K  Ries,  University  of 

Suicide  Prevention:  Ph.D.  Washington 

Building  Skills, 

Resources  and 
Capacity 


Civilians 


Eunice  09/09- 

Kennedy  08/10 

Shriver 

National 

Institute  on 

Child  Health 

and  Human 

Development 


421,116  School-Based  Elaine  M.  Walsh,  University  of 

Suicide  Prevention:  Ph.D.  Washington 

Building  Skills, 

Resources  and 
Capacity 


Civilians 


Eunice  07/10- 

Kennedy  05/14 

Shriver 

National 

Institute  on 

Child  Health 

and  Human 

Development 


1,014,831  Quasi-Experimental  Brian  M.  Indiana  University,  Civilians 

Studies  of  Early  Risk  D'Onofrio,  Ph.D.  Bloomington 
Factors  for  Severe 
Psychopathology 


Risk  and  protective  factor 
interactions 

Prediction  of  imminent  risk 
Population-based  screening 


Enhanced  continuity  of  care 
Provider  and  gatekeeper 
training 

Risk  and  protective  factor 
interactions 


Enhanced  continuity  of  care 

Provider  and  gatekeeper 
training 

Risk  and  protective  factor 
interactions 


Risk  and  protective  factor 
interactions 


MOMRP  Category 


Early  screening/intervention 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 


Prevention,  education,  and 
training 

Early  screening/intervention 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 


Prevention,  education,  and 
training 

Early  screening/intervention 
Epidemiology  and/or 
basic  science/neurological 
mechanisms 


Early  screening/intervention 
Epidemiology  and/or 
basic  science/neurological 
mechanisms 
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Table  A. 4 — Continued 


Funding  Start-  Principal 

Dept.  End  Funding  (S)  Title  Investigator(s) 


National 
Institute  on 
Drug  Abuse 

05/09- 

04/10 

195,000 

Preventing 

Addiction-Related 

Suicide 

Richard  Ries, 

M.D. 

National 
Institute  on 
Drug  Abuse 

01/09- 

05/10 

193,125 

Developing  an 
Intervention  to 
Address  Suicide  Risk 
During  Substance 
Use  Disorder 

Mark  llgen,  Ph.D. 

NIMH 

07/03- 

03/08 

1,324,991 

Community- 
Based  Cognitive 
Therapy  for  Suicide 
Attempters 

Aaron  T.  Beck, 
M.D. 

NIMH 

09/03- 

08/09 

584,760 

Study  of 

Gatekeeper 

Training  for  Suicide 
Prevention 

Peter  A.  Wyman, 
Ph.D. 

NIMH 

08/06- 

05/11 

2,028,646 

Components  of 
Effective  Suicide 
Prevention  in  the 

Kerry  L.  Knox, 
Ph.D. 

USAF 


Researching 

Organization 


University  of 
Washington 


University  of 
Michigan 


University  of 
Pennsylvania 


University  of 
Rochester 


University  of 
Rochester 


00 


Participants  NAASP  Category  MOMRP  Category 


Civilians 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Risk  and  protective  factor 
interactions 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

Civilians 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Risk  and  protective  factor 
interactions 

Treatment 

Civilians 

Prevention  of  reattempts 

Psychosocial  interventions 
for  those  at  risk 

Assessment 

Treatment 

Civilians 

Provider  and  gatekeeper 
training 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Risk  and  protective  factor 
interactions 

Prevention,  education,  and 
training 

Early  screening/intervention 

Assessment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

Air  Force 

Provider  and  gatekeeper 
training 

Psychosocial  interventions 
for  those  at  risk 

Risk  and  protective  factor 
interactions 

Stigma  reduction 

Population-based  risk 
reduction/resilience-building 

Prevention,  education,  and 
training 

Early  screening/intervention 
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Table  A. 4 — Continued 


Funding 

Dept. 

Start- 

End 

Funding  (S) 

Title 

Principal 

Investigator(s) 

Researching 

Organization 

NIMH 

03/07- 

02/10 

635,842 

Development  of 
an  Intervention  for 
Suicide  Attempters 

Ivan  W.  Miller, 
Ph.D. 

Butler  Hospital 

NIMH 

05/07- 

02/10 

625,726 

Family  Based 
Intervention  for 
Adolescent  Suicide 
Attempters 

Joan  R. 

Asarnow,  Ph.D. 

University  of 
California,  Los 
Angeles 

NIMH 

08/07- 

05/10 

615,600 

Adolescent 

Emergency 

Patients:  Suicide 

Risk  Detection 
and  Treatment 
Facilitation 

Cheryl  A.  King, 
Ph.D. 

University  of 
Michigan 

NIMH 

03/08- 

12/12 

2,177,675 

5-HTiA  Receptor 
Anti-Apoptotic 
Transduction 
Pathways  in  Suicide 

Victoria  Arango, 
Ph.D. 

Columbia 
University  Health 
Sciences 

NIMH 

09/08- 

05/13 

3,261,056 

Prevention  of 
Suicidality  in 

College  Students: 

A  Common  Core 
Process  Approach 

Steven  C.  Hayes, 
Ph.D. 

University  of 
Nevada,  Reno 

NIMH 

08/09- 

07/12 

725,838 

Concurrent 
Treatment  for 
Parents  and 
Adolescents  Who 
Attempt  Suicide 

Anthony  N. 
Spirito,  Ph.D. 

Brown  University 

NIMH 

09/09- 

05/14 

9,872,609 

Emergency 

Department 

Safety  Assessment 
and  Follow-Up 
Evaluation  (ED-SAFE, 
Trial 

Edwin  D. 

Boudreaux,  Ph.D. 

1 

University  of 
Massachusetts 

Participants 


NAASP  Category 


MOMRP  Category 


Civilians 


Civilians 


Civilians 


Civilians 


Civilians 


Civilians 


Civilians 


Prevention  of  reattempts 
Prediction  of  imminent  risk 

Prevention  of  reattempts 


Provider  and  gatekeeper 
training 

Prediction  of  imminent  risk 


Improved  biological 
interventions 


Psychosocial  interventions 
for  those  at  risk 


Prevention  of  reattempts 
Prediction  of  imminent  risk 


Treatment 


Treatment 


Assessment 
Epidemiology  and/or 
basic  science/neurological 
mechanisms 


Epidemiology  and/or 
basic  science/neurological 
mechanisms 


Early  screening/intervention 


Treatment 


Enhanced  continuity  of  care  Early  screening/intervention 
Prediction  of  imminent  risk  Assessment 

Treatment 
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Table  A. 4 — Continued 


Funding 

Dept. 

Start- 

End 

Funding  (S) 

Title 

Principal 

Investigator(s) 

Researching 

Organization 

Participants 

NAASP  Category 

MOMRP  Category 

NIMH 

07/10- 

04/15 

1,749,266 

Effectiveness  Trial 
of  Youth  Suicide 
Prevention 

Delivered  by  Teen 
Peer  Leaders 

Peter  A.  Wyman, 
Ph.D. 

University  of 
Rochester 

Civilians 

Psychosocial  interventions 
for  those  at  risk 

Stigma  reduction 
Population-based  screening 

Early  screening/intervention 

Treatment 

NIMH 

02/10- 

01/14 

566,200 

Measurement  of 
Social-Cognitive 

Risk  Factors  for 
Suicidal  Ideation 
and  Behavior 

Regina  Miranda, 
Ph.D. 

City  University 
of  New  York, 
Research 
Foundation 

Civilians 

Risk  and  protective  factor 
interactions 

Prediction  of  imminent  risk 

Assessment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

NIMH 

04/12- 

08/16 

1,289,112 

Attachment-Based 
Family  Therapy  for 
Suicidal  Adolescents 

Guy  S.  Diamond, 
Ph.D. 

Children's  Hospital  Civilians 
of  Philadelphia 

Prevention  of  reattempts 

Provider  and  gatekeeper 
training 

Treatment 

NIMH 

10/12- 

12/12 

485,671 

Coping  Long-Term 
with  Attempted 
Suicide  Program — 
Adolescents 

Shirley  Yen,  Ph.D.  Brown  University 

Civilians 

Prevention  of  reattempts 
Risk  and  protective  factor 
interactions 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

NIMH 

09/12- 

08/16 

235,500 

Reducing  Suicide 
Ideation  Through 
Insomnia  Treatment 
(REST-IT) 

Andrew  D. 
Krystal,  Ph.D. 

Duke  University 

Civilians 

Improved  biological 
interventions 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

NIMH 

07/12- 

04/17 

584,104 

Ketamine  vs. 
Midazolam:  Testing 
Rapid  Relief  of 
Suicide  Risk  in 
Depression 

Michael  F. 

Grunebaum, 

Ph.D. 

Columbia 

University 

Civilians 

Risk  and  protective  factor 
interactions 

Improved  biological 
interventions 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

NIMH 

07/12- 

08/13 

360,000 

Pilot  Study  of 

Online 

Interventions  for 
Population-Based 
Suicide  Prevention 

Ursula  S. 
Whiteside,  Ph.D. 

Group  Health 
Cooperative 
Central  Hospital 

Civilians 

Population-based  risk 
reduction/resilience¬ 
building 

Prediction  of  imminent  risk 

Population-based  screening 

Early  screening/intervention 
Epidemiology  and/or 
basic  science/neurological 
mechanisms 
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Table  A. 4 — Continued 


Funding 

Dept. 

Start- 

End 

Funding  (S) 

Title 

Principal 

Investigator(s) 

Researching 

Organization 

Participants 

NAASP  Category 

NIMH 

09/07- 

05/11 

295,200 

Genetic  Risk  Factor 
Suicidal  Behavior 

Virginia  L. 
Willour,  Ph.D. 

University  of  Iowa 

Civilians 

Risk  and  protective  factor 
interactions 

NIMH 

12/09- 

11/14 

642,170 

Stress  Responses 
as  Prospective 
Predictors  of  Girls' 
Suicidality  and  Self- 
Injury 

Mitchell  J. 
Prinstein,  Ph.D. 

University  of  North  Civilians 
Carolina  Chapel 

Hill 

Risk  and  protective  factor 
interactions 

NIMH 

12/08- 

11/09 

400,750 

Future  Suicide 
Attempt: 
Psychobiological 
Features 

Maria  A. 
Oquendo,  M.D. 

Columbia 

University 

Civilians 

Risk  and  protective  factor 
interactions 

NIMH 

04/09- 

02/10 

244,383 

Adapting  Exercise 
Treatment  for 
Depression  to 
Adolescents:  A  Pilot 
Study 

Andrea  L.  Dunn, 
Ph.D. 

Klein  Buendel,  Inc. 

Civilians 

Provider  and  gatekeeper 
training 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Risk  and  protective  factor 
interactions 

NIMH 

07/09- 

06/10 

129,546 

Statistical  Methods 
for  Predicting 
Suicide  Attempt 

Hanga  Galfaivy, 
Ph.D. 

Columbia 

University 

Civilians 

Prediction  of  imminent  risk 

NIMH 

08/09- 

04/12 

178,340 

Antidepressant 

Use  and  Suicidality: 
Comparative 

Safety  in  Children 
and  Adults 

Sebastian 

Schneeweiss, 

M.D. 

Brigham  and 
Women's  Hospital 

Civilians 

Risk  and  protective  factor 
interactions 

Reduction  in  access  to  lethal 

means 

MOMRP  Category 


Epidemiology  and/or 
basic  science/neurological 
mechanisms 

Early  screening/intervention 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

Early  screening/intervention 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

Treatment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 


Prevention,  education,  and 
training 

Early  screening/intervention 
Epidemiology  and/or 
basic  science/neurological 
mechanisms 

Prevention,  education,  and 
training 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 
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Table  A. 4 — Continued 


Funding  Start-  Principal  Researching 

Dept.  End  Funding  (S)  Title  Investigator(s)  Organization  Participants  NAASP  Category  MOMRP  Category 


NIMH 

08/09- 

01/12 

265,558 

Antidepressant  Use 
and  Suicide 

Susan  Busch, 
Ph.D. 

Yale  University 

Civilians 

Risk  and  protective  factor 
interactions 

Reduction  in  access  to  lethal 

means 

Prevention,  education,  and 
training 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

NIMH 

07/09- 

06/10 

32,150 

Anorexia  Nervosa 
and  Suicidal 
Behaviors 

April  Smith, 

Ph.D. 

Florida  State 
University 

Civilians 

Risk  and  protective  factor 
interactions 

Prediction  of  imminent  risk 

Assessment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

NIMH 

09/09- 

08/10 

606,836 

Glutamatergic 
Modulators  for 
Rapid  and 

Sustained 

Antidepressant 

Effect 

April  Smith, 

Ph.D. 

Florida  State 
University 

Civilians 

Improved  biological 
interventions 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

NIMH 

01/07- 

01/09 

286,771 

Seasonality  of 
Suicide  and 
Airborne  Allergens 

Teodor 

Postolache,  M.D. 

University  of 
Maryland 

Civilians 

Risk  and  protective  factor 
interactions 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

NIMH 

09/11- 

09/12 

41,309 

Early  Stress  and 
Suicidal  Behavior 

Quetzal  Class 

Indiana  University, 
Bloomington 

Civilians 

Prediction  of  imminent  risk 

Prevention,  education,  and 
training 

Early  screening/interventior 

NIMH 

08/11- 

08/12 

41,800 

Brief  Skills  Training 
Intervention  for 
Suicidal  Individuals 

Erin  Ward- 
Ciesielski 

University  of 
Washington 

Civilians 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions 
for  those  at  risk 

Treatment 

NIMH 

07/11- 

05/16 

614,555 

Family  Therapy  for 
Hospital  Care 

Guy  Diamond, 
Ph.D. 

Children's  Hospital  Civilians 
of  Philadelphia 

Enhanced  continuity  of  care 
Psychosocial  interventions 
for  those  at  risk 

Treatment 
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Table  A. 4 — Continued 


Funding 

Dept. 

Start- 

End 

Funding  (S)  Title 

Principal 

Investigator(s) 

Researching 

Organization 

Participants 

NAASP  Category 

MOMRP  Category 

National 
Institute 
of  Nursing 
Research 

09/04- 

08/08 

1,364,937  Youth  Suicide 
Prevention: 
Maintaining  Long- 
Term  Change 

Brooke  P. 

Randell,  Ph.D. 
(year  1-3); 

Carole  Hooven, 
Ph.D.  (year  4) 

University  of 
Washington 

Civilians 

Population-based  risk 
reduction/resilience¬ 
building 

Early  screening/intervention 

Assessment 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 

National 
Institute 
of  Nursing 
Research 

06/06- 

03/11 

1,606,542  Cast  Intervention 
for  Reducing 
Adolescent  Risk 
Behaviors 

Elaine  A. 
Thompson, 

Ph.D.,  R.N. 

University  of 
Washington 

Civilians 

Risk  and  protective  factor 
interactions 

Prevention,  education,  and 
training 

Epidemiology  and/or 
basic  science/neurological 
mechanisms 
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Table  A.5 

American  Foundation  for  Suicide  Prevention  Studies 


Grant 

Year 

Funding  (S) 

Title 

Principal 

Investigator(s) 

Researching 

Organization 

2012 

100,000 

Mindfulness-Based 
Cognitive  Therapy  + 

Safety  Planning  for 

Suicidal  Behavior:  A 
Treatment  Development 
Study  Postdoctoral 
Research  Fellowship 

Megan  Chesin, 
Ph.D. 

Columbia 

University 

2012 

98,804 

Feasibility  of  an  Online 
Intervention  for 
Population-Based  Suicide 

Ursula 

Whiteside, 

Ph.D. 

Group  Health 
Cooperative 

2012 

85,000 

Self  Harm  and  Suicide 
Attempt  Risk:  Evaluation 
of  an  Intervention  for 
Youths  with  Self-Harm 
Behavior 

Jennifer  L. 
Hughes,  Ph.D. 

University  of 
California,  Los 
Angeles 

2012 

85,000 

Optical  Brain  Imaging 
Predictors  of  Treatment 
Response  to  Dialectical 
Behaviour  Therapy  for 
Non-Suicidal  Self-Injury 
in  Borderline  Personality 
Disorder 

Anthony 

Ruocco,  Ph.D. 

University 
of  Toronto, 
Scarborough 

2012 

85,000 

Identifying  Risk  Factors 
and  Intervention  Methods 
to  Prevent  Suicide  in 
Pediatric  Bipolar  Disorder 

Sally  M. 
Weinstein, 

Ph.D. 

University  of 
Illinois  at  Chicago 

2012 

84,072 

The  Electronic 

Intervention  for 

Michael  Armey, 
Ph.D. 

Butler  Hospital 

Suicidality  (EIS):  A  Novel 
Adjunctive  Treatment  for 
Suicide  Risk 


Participants  NAASP  Category  MOMRP  Category 


Civilians 


Psychosocial  interventions  for  Assessment 

those  at  risk  Treatment 

Epidemiology  and/or  basic 
science/neurological  mechanisms 


Civilians 


Civilians 


Civilians 


Psychosocial  interventions  for 
those  at  risk 

Risk  and  protective  factor 
interactions 

Enhanced  continuity  of  care 
Risk  and  protective  factor 
interactions 


Prediction  of  imminent  risk 
Improved  biological 
interventions 


Treatment 


Prevention,  education,  and 

training 

Assessment 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

Recovery  and  return  to  duty 
Epidemiology  and/or  basic 
science/neurological  mechanisms 


Civilians  Prediction  of  imminent  risk  Prevention,  education,  and 

training 

Early  screening/intervention 

Assessment 

Treatment 

Civilians  Enhanced  continuity  of  care  Prevention,  education,  and 

training 

Treatment 
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Table  A. 5 — Continued 


Grant  Principal  Researching 

Year  Funding  (S)  Title  Investigator(s)  Organization  Participants  NAASP  Category  MOMRP  Category 


2012 

2012 

2012 

2012 

2012 

2012 

2012 


83,911 

Posttraumatic  Stress 
Disorder  and  Suicide 
Among  Massachusetts 
Veterans 

Jaimie  L.  Gradus, 
D.Sc.,  M.P.H. 

Boston  VA 
Research  Institute 

Veterans 

Enhanced  continuity  of  care 

Prevention,  education,  and 
training 

75,000 

Placebo-Controlled  Trial 
of  Lithium  +  TAU  for  Acute 
Suicidal  Ideation  and/or 
Behavior  in  Patients  with 
Major  Depression 

Michael  Bauer, 
M.D.,  Ph.D. 

University  Hospital  Civilians 
Carl  Gustav 

Carus,  Technische 

Universitat 

Dresden 

Prevention  of  reattempts 
Improved  biological 
interventions 

Treatment 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

75,000 

Modeling  5-HT1A 

Receptor  Transduction 
Pathways  in  Suicide 

Thomas  Franke, 
M.D.,  Ph.D. 

New  York 
University  School 
of  Medicine 

Civilians 

Psychosocial  interventions  for 
those  at  risk 

Risk  and  protective  factor 
interactions 

Population-based  risk 
reduction/resilience-building 

Prevention,  education,  and 
training 

Early  screening/intervention 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

75,000 

Multilingual  Assessment 
of  Suicidal  Ideation  and 
Behavior  on  the  Internet 

Yan  Leykin, 

Ph.D. 

University  of 
California,  San 
Francisco 

Civilians 

Population-based  risk 
reduction/resilience-building 

Prediction  of  imminent  risk 

Treatment 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

75,000 

Brief  Sleep  Intervention 
for  Suicide  In  Bipolar 
Disorder 

Louisa  Sylvia, 
Ph.D. 

Massachusetts 
General  Hospital 

Civilians 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions  for 
those  at  risk 

Prevention,  education,  and 
training 

Assessment 

Treatment 

74,944 

Suicide  Onset  Foliowing 
Antidepressant  Initiation 

Robert  Penfold, 
Ph.D. 

Group  Health 
Cooperative 

Civilians 

Psychosocial  interventions  for 
those  at  risk 

Treatment 

74,767 

Randomized,  Double- 
Blind  Ketamine 
Augmentation  in 
Chronically  Suicidal, 
Treatment-Resistant  Major 
Depression 

Cristina  Cusin, 
M.D. 

Massachusetts 
General  Hospital 

Civilians 

Psychosocial  interventions  for 
those  at  risk 

Risk  and  protective  factor 
interactions 

Population-based  risk  reduction/ 
resilience-building 

Treatment 

Prediction  of  imminent  risk 
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Table  A. 5 — Continued 


Grant  Principal  Researching 

Year  Funding  (S)  Title  Investigator(s)  Organization  Participants 


2012 

72,818 

Improving  Clinical 
Evaluations  of  Suicide  Risk 
and  Their  Relationship  to 
Care  Planning 

Christopher 

Buckingham, 

Ph.D. 

Aston  University, 
UK 

Civilians 

2012 

72,040 

Feasibility  and  Efficacy 
of  a  DBT  Skills  "App"  for 
Suicidal  Individuals  with 
BPD 

Shireen  Rizvi, 
Ph.D. 

Rutgers  University 

Civilians 

2012 

30,000 

Familial  Risk  for  Suicide 
and  Self-Injury: 

Testing  Theories  in 
Multigenerational 
Pedigrees 

Sheila  Crowell, 
Ph.D. 

University  of  Utah 

Civilians 

2012 

30,000 

A  Pilot  Study  of  Suicide 
Assessment  in  Rural 

Adults 

Kelly  Cukrowicz, 
Ph.D. 

Texas  Tech 
University 

Civilians 

2012 

30,000 

Person-Specific  Modeling 
of  Suicidal  Ideation  in 
Sexual  Minority  Youth 

Katerina  0. 
Sinclair,  Ph.D., 
M.A.S. 

University  of 
Arizona 

Civilians 

2012 

29,969 

Medical  Students'  Suicide 
Risk  Assessment 
Proficiency  After 
Interacting  with  a  Virtual 
Patient  in  Crisis 

Adriana  Foster, 
M.D. 

Medical  College  of  Civilians 
Georgia,  Georgia 

Health  Sciences 

University 

2011 

100,000 

Life  Stress  and  Affect 
Regulation  in  Multiple 
and  First-Time  Adolescent 
Suicide  Attempters 

Richard  Liu, 

M.A.,  Ph.D. 
candidate 

University  of 
Illinois,  Chicago 

Civilians 

2011 

100,000 

White  Matter  Integrity 
and  Suicide  Attempt 
History 

Doreen  Olvet, 
Ph.D. 

Columbia 

University 

Civilians 

NAASP  Category 

MOMRP  Category 

Prevention  of  reattempts 

Assessment 

Psychosocial  interventions  for 
those  at  risk 

Risk  and  protective  factor 
interactions 

Prevention,  education,  and 
training 

Assessment 

Prediction  of  imminent  risk 

Prevention,  education,  and 
training 

Affordable,  accessible,  and 
effective  care 

Prediction  of  imminent  risk 

Prevention,  education,  and 
training 

Prediction  of  imminent  risk 

Assessment 

Prediction  of  imminent  risk 

Prevention,  education,  and 
training 

Assessment 

Improved  biological 
interventions 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

Psychosocial  interventions  for 
those  at  risk 

Prevention,  education,  and 
training 

Assessment 

Treatment 
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Table  A. 5 — Continued 


Grant  Principal  Researching 

Year  Funding  (S)  Title  Investigator(s)  Organization  Participants  NAASP  Category  MOMRP  Category 


2011 

88,000 

Functional  Analysis 
of  Differential  DNA 
Methylatlon  In  Frontal 
Cortex  of  Suicide 
Completers 

Gilles  Maussion,  Douglas  Mental 
Ph.D.  Health  Institute, 

McGill  University 

Civilians 

Risk  and  protective  factor 
interactions 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

2011 

85,000 

Lethal  Means  Restriction 
for  Suicide  Prevention: 
Beliefs  and  Behaviors  of 
Emergency  Department 
Providers 

Marian  Betz,  University  of 

M.D.,  M.P.H.  Colorado,  Denver 

Civilians 

Psychosocial  interventions  for 
those  at  risk 

Prevention,  education,  and 
training 

Assessment 

2011 

85,000 

Copy  Number  Variation  in 
Suicide 

Carl  Ernst,  Ph.D.  Harvard 

University 

Civilians 

Enhanced  continuity  of  care 
Stigma  reduction 

Prevention,  education,  and 
training 

Early  screening/intervention 

2011 

85,000 

A  Test  of  the 

Interpersonal- 
Psychological  Theory  of 
Suicide  in  Prison  Inmates 

Jon  T.  University 

Mandracchia,  of  Southern 

Ph.D.  Mississippi 

Civilians 

Improved  biological 
interventions 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

2011 

85,000 

Ketamine  for  Rapid 
Reduction  of  Suicidal 
Ideation  and  Suicide  Risk  in 
Flospitalized  Patients 

James  W.  Mount  Sinai 

Murrough,  M.D.  School  of  Medicine 

Civilians 

Psychosocial  interventions  for 
those  at  risk 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

2011 

85,000 

Objective  Sleep  and  Suicide  Ravi  Singareddy,  Pennsylvania 

In  General  Population  M.D.  State  University 

Sample  of  Youth 

Civilians 

Risk  and  protective  factor 
interactions 

Prevention,  education,  and 
training 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

2011 

81,990 

The  Acquired  Capability 
for  Suicide:  A  Mechanism 
Underlying  the  Gender 
Disparity  in  Suicide  Rates 

Phillip  Smith,  University  of 

Ph.D.  South  Alabama 

Civilians 

Risk  and  protective  factor 
interactions 

Early  screening/intervention 
Epidemiology  and/or  basic 
science/neurological  mechanisms 

2011 

75,000 

The  Neural  Circuitry  of 
Suicidality  in  Adolescent 
Depression 

Hilary  P.  Yale  University 

Blumberg,  M.D. 

Civilians 

Psychosocial  interventions  for 
those  at  risk 

Epidemiology  and/or  basic 
science/neurological  mechanisms 
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Table  A. 5 — Continued 


Grant 

Year 

Funding  (S) 

Title 

Principal 

Investigator(s) 

Researching 

Organization  Participants 

NAASP  Category 

MOMRP  Category 

2011 

75,000 

Meta-Analysis  of  Case- 
Control  Psychological 
Autopsy  Studies  of  Suicide 

Kenneth  R. 
Conner,  Psy.D. 

University  of  Civilians 

Rochester  Medical 

Center 

Improved  biological 
interventions 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

2011 

75,000 

The  Consequences  of 
Bereavement  by  Suicide 

Annette 
Erlangsen,  Ph.D. 

Johns  Hopkins  Civilians 

University 

Risk  and  protective  factor 
interactions 

Stigma  reduction 

Prediction  of  imminent  risk 

Postvention 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

2011 

75,000 

Fatal  Decisions:  Behavioral 
Economic  Study  of 

Decision  Pathways  to 
Suicidal  Behavior  in  the 
Second  Half  of  Life 

Katalin  Szanto, 
M.D. 

University  of  Civilians 

Pittsburgh  Western 

Psychiatric  Institute 

Psychosocial  interventions  for 
those  at  risk 

Prevention,  education,  and 
training 

Early  screening/intervention 

2011 

74,655 

Smoking  and  Suicide: 
Changes  in  State  Tobacco 
Policies  as  Natural 
Experiments 

Richard  A. 
Grucza,  Ph.D., 
M.P.E. 

Washington  Civilians 

University  School 
of  Medicine 

Prediction  of  imminent  risk 

Prevention,  education,  and 
training 

Assessment 

2011 

73,680 

A  Functional  Magnetic 
Imaging  Study  (fMRI)  in 
First-Degree  Relatives  of 
Suicide  Completers 

Fabrice  Jollant, 
M.D.,  Ph.D. 

Douglas  Mental  Civilians 

Health  Institute, 

McGill  University 

Psychosocial  interventions  for 
those  at  risk 

Improved  biological 
interventions 

Early  screening/intervention 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

2010 

100,000 

Investigation  of 
Inheritance  Patterns 
of  Polymorphisms 
in  Genes  Regulating 
the  Hypothalamic- 
Pituitary-Adrenal  (HPA 
Axis),  in  Relation  to 

Suicidal  Behavior  and  Its 
Endophenotypes 

Danuta 

Wasserman, 

M.D.,  Ph.D. 

Karolinska  Civilians 

Institute, 

Stockholm,  Sweden 

Psychosocial  interventions  for 
those  at  risk 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

2010 

100,000 

GABA  System  Genes 
and  Suicidal  Behavior  in 
Psychiatric  Disorders 

Clement  Zai, 
Ph.D. 

Centre  for  Civilians 

Addiction  and 

Mental  Health, 

Toronto 

Risk  and  protective  factor 
interactions 

Assessment 

Epidemiology  and/or  basic 
science/neurological  mechanisms 
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Grant  Principal  Researching 

Year  Funding  (S)  Title  Investigator(s)  Organization  Participants  NAASP  Category  MOMRP  Category 


2010 

85,000 

Global  Expression 

Analysis  of  Patients  with 
Treatment  Emergent 
Suicidal  Ideations 

Young  Investigator  Grant 

Falk  Lohoff,  M.D.  University  of 
Pennsylvania 

Civilians 

Improved  biological 
interventions 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

2010 

84,888 

Identification  of 
Neurophysiological 
Markers  of  Suicidal 
Behavior  and  Impulsivity 
in  Bipolar  Disorder 

Masoud  Kamali, 
M.D. 

University  of 
Michigan 

Civilians 

Risk  and  protective  factor 
interactions 

Assessment 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

2010 

74,944 

MicroRNAs  in  Postmortem 
Brains  of  Suicide  Subjects 

Yogesh  Dwivedi, 
Ph.D. 

University  of 
Illinois  at  Chicago 

Civilians 

Improved  biological 
interventions 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

2009 

240,031 

Optimizing  Treatment  of 
Complicated  Grief 

Katherine  Shear, 
M.D. 

Columbia 

University 

University  of 
Pittsburgh  School 
of  Medicine 

Harvard  Medical 
School 

Civilians 

Enhanced  continuity  of  care  Treatment 

Provider  and  gatekeeper  Epidemiology  and/or  basic 

training  science/neurological  mechanisms 

Affordable,  accessible,  and 
effective  care 

Improved  biological  interventions 

University  of 
California,  San 
Diego 

San  Diego  VA 
Healthcare  System 

2009 

99,996 

Emergency  Mental  Health 
Management  Following 
Deliberate  Self-Harm 

Mark  Olfson, 
M.D.,  M.P.H. 

Columbia 

University 

Civilians 

Enhanced  continuity  of  care 

Affordable,  accessible,  and 
effective  care 

Treatment 

2009 

99,500 

Differential  microRNA 
(pRNA)  Expression  in 
the  Prefrontal  Cortex  of 
Suicides 

Gustavo  Turecki, 
M.D.,  Ph.D. 

Douglas  Hospital 
Research  Institute, 
McGill  University 

Civilians 

Improved  biological 
interventions 

Epidemiology  and/or  basic 
science/neurological  mechanisms 
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Table  A. 5 — Continued 


Grant  Principal  Researching 

Year  Funding  (S)  Title  Investigator(s)  Organization 


2009 

98,570 

An  fMRI  Study  of  Affect 
Arousal  and  Cognitive 
Control  in  Suicidal  Subjects 
with  Borderline  Personality 
Disorder 

Paul  Soloff, 

M.D. 

University  of 
Pittsburgh 

2009 

98,406 

Factors  Associated  with 
Suicide  in  Youth  and 

Adults  with  Eating 
Disorders 

Cynthia  Bulik, 
Ph.D.,  F.A.E.D. 

University  North 
Carolina 

2009 

85,000 

Personality  Disorders 
and  Suicidal  Behaviors: 

A  Prospective  Study  of 
Associations,  Mediators 
and  Moderators 

Emily  B.  Ansell, 
Ph.D. 

Yale  University 
School  of  Medicin 

2009 

85,000 

Investigation  into  the 

Role  of  Genes  and  Stress 
in  Depression  and  Suicide 
Among  Medical  Interns 

Srijan  Sen,  Ph.D., 
M.D. 

University  of 
Michigan 

2009 

85,000 

BDNF  Promoter 
Methylation  and  Suicidal 
Behavior  in  Bipolar 
Disorder 

John  Strauss, 
M.D.,  M.Sc. 

Centre  for 
Addiction  and 
Mental  Health, 
University  of 
Toronto 

2009 

85,000 

Development  of  a 
Treatment  Protocol 
for  Suicidal  Latino/a 
Adolescents 

Yovanska 

Duarte-Velez, 

Ph.D. 

Brown  University 

2009 

84,722 

Epidemiology  of 
Undetermined  Deaths: 

Nathalie 

Huguet,  Ph.D. 

Portland  State 
University 

Prevalence  and  Patterns  of 
Misclassified  Suicides 


Participants  NAASP  Category  MOMRP  Category 


Civilians 


Psychosocial  interventions  for 
those  at  risk 


Epidemiology  and/or  basic 
science/neurological  mechanisms 


Civilians 


Risk  and  protective  factor 
interactions 

Prediction  of  imminent  risk 


Assessment 

Epidemiology  and/or  basic 
science/neurological  mechanisms 


Civilians 


Psychosocial  interventions  for 
those  at  risk 


Treatment 

Epidemiology  and/or  basic 
science/neurological  mechanisms 


Civilians  Improved  biological 

interventions 


Epidemiology  and/or  basic 
science/neurological  mechanisms 


Civilians  Improved  biological 

interventions 


Epidemiology  and/or  basic 
science/neurological  mechanisms 


Civilians  Provider  and  gatekeeper  Treatment 

training 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions  for 
those  at  risk 

Prediction  of  imminent  risk 

Civilians  Risk  and  protective  factor  Epidemiology  and/or  basic 

interactions  science/neurological  mechanisms 
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Grant  Principal  Researching 

Year  Funding  (S)  Title  Investigator(s)  Organization  Participants  NAASP  Category  MOMRP  Category 


2009 

84,508 

Evaluation  of  a 

Gatekeeper  Training 
Program  as  Suicide 
Intervention  Training  for 
Medical  Students 

Shay-Lee  Bolton, 
M.Sc. 

University  of  Civilians 

Manitoba 

Enhanced  continuity  of  care 

Provider  and  gatekeeper 
training 

Prevention,  education,  and 
training 

2009 

75,000 

Predicting  Suicidal 
Behavior:  Internal 
Consistency  and  Predictive 
Validity  of  the  Suicide 
Trigger  Scale 

Igor  Galynker, 
M.D.,  Ph.D. 

Beth  Israel  Civilians 

Medical  Center 

Prediction  of  imminent  risk 

Prevention,  education,  and 
training 

Early  screening/intervention 

2009 

75,000 

Paliperidone  and  Lithium 
in  the  Treatment  of 
Suicidality-Treatment 
Indication  and  Epigenetic 
Regulation 

Richard  Shelton, 
M.D. 

University  Civilians 

of  Alabama, 

Birmingham 

Enhanced  continuity  of  care 

Improved  biological 
interventions 

Treatment 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

2009 

75,000 

Number  and  Severity 
of  Suicide  Attempts: 
Relationship  with 
Toxoplasma  Gondii 
Antibodies 

Teodor 

Postolache,  M.D. 

University  of  Civilians 

Maryland  School  of 

Medicine 

Improved  biological 
interventions 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

2009 

74,700 

A  Randomized  Controlled 
Trial  Investigating  the 
Impact  of  a  Regular  SMS 
on  Suicide  Risk  Among 
School  Students 

Jo  Robinson, 
M.Sc. 

Orygen  Youth  Civilians 

Health  Research 

Center 

Enhanced  continuity  of  care 

Psychosocial  interventions  for 
those  at  risk 

Treatment 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

2009 

74,526 

Genetic  and  Clinical 
Predictors  of  Suicidal 
Behavior  in  Veterans 
Returning  from  Iraq  and 
Afghanistan 

Rachel  Yehuda, 
Ph.D. 

Mount  Sinai  Veterans 

School  of  Medicine 

Risk  and  protective  factor 
interactions 

Early  screening/intervention 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

2009 

29,320 

Alcohol  as  an  Acute  Risk  Courtney  Bagge, 

Factor  for  Suicide  Ph.D. 

Attempts:  A  Case-Crossover 

Pilot  Study 

University  of  Civilians 

Mississippi  Medical 

Center 

Risk  and  protective  factor 
interactions 

Early  screening/intervention 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

Ongoing  Studies  of  Relevance  to  Suicide  Prevention  Among  Military  Personnel  131 


Table  A. 5 — Continued 


Grant  Principal  Researching 

Year  Funding  (S)  Title  Investigator(s)  Organization 


2008 

100,000 

Diffusion  Tensor  Imaging  Emil  F.  Coccaro, 
Studies  of  Suicidal  Behavior  M.D. 

University  of 
Chicago 

2008 

100,000 

Childhood  Traumatic 

Stress  and  Adolescent 

Girls'  Suicidality:  A 
Longitudinal  Examination 
of  Psychophysiological 
Mechanisms 

Nicole  Heilbron, 
Ph.D. 

University  of 

North  Carolina 

2008 

100,000 

Dissecting  Serotonergic 
Influences  on  Impulsivity 
and  Aggression 

Luis  Pennanen, 
Ph.D. 

University  of 
California, 

San  Francisco 

2008 

100,000 

Examination  of  a  Relapse 
Prevention  Task  in 
Preventing  Future  Suicide 
Attempts 

Megan  Spokas, 
Ph.D. 

University  of 
Pennsylvania 

2008 

99,998 

Suicide  Clusters  Across  the 
Globe:  Geospatial  Trends 
from  America,  Ireland 
and  New  Zealand 

Gregory  Luke 
Larkin,  M.D., 
M.S.,  M.S.P.H. 

Yale  University 
School  of  Medicine 

2008 

99,928 

Safety  Planning 
Intervention  for  Suicidal 
Individuals  Who  Contact 
Crisis  Centers 

Barbara  Stanley, 
Ph.D. 

Columbia 

University 

2008 

89,783 

A  Prospective  Study  of 
Completed  Suicide  in  a 
Large  Bipolar  1  Disorder 
Sample 

William  Coryell, 
M.D. 

University  of 

Iowa 

2008 

85,000 

Bio-Behavioral  Markers  of 
Affect  Regulation  in  Teen 

Daniel  P. 
Dickstein,  M.D. 

Brown  University 

Suicide  Attempters  versus 
Non-Suicidal  Self-Injurers 


Participants  NAASP  Category  MOMRP  Category 


Civilians 


Civilians 


Psychosocial  interventions  for 
those  at  risk 

Risk  and  protective  factor 
interactions 

Prediction  of  imminent  risk 


Prevention,  education,  and 
training 

Epidemiology  and/or  basic 
science/neurological  mechanisms 


Civilians 


Civilians 


Civilians 


Prediction  of  imminent  risk 

Improved  biological 
interventions 

Enhanced  continuity  of  care 
Provider  and  gatekeeper 
training 

Psychosocial  interventions  for 
those  at  risk 

Risk  and  protective  factor 
interactions 


Assessment 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

Postvention 

Epidemiology  and/or  basic 
science/neurological  mechanisms 


Epidemiology  and/or  basic 
science/neurological  mechanisms 


Civilians  Provider  and  gatekeeper  Treatment 

training 

Psychosocial  interventions  for 
those  at  risk 

Civilians  Population-based  risk  reduction/  Postvention 

resilience-building  Epidemiology  and/or  basic 

Prediction  of  imminent  risk  science/neurological  mechanisms 


Civilians 


Prediction  of  imminent  risk 
Improved  biological 
interventions 


Early  screening/intervention 
Epidemiology  and/or  basic 
science/neurological  mechanisms 
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Table  A. 5 — Continued 


Grant 

Year 

Funding  (S) 

Title 

Principal 

Investigator(s) 

Researching 

Organization 

2008 

85,000 

Cognitive  Control  and 
Emotion  Processing 
Markers  of  High-Intent 
Suicide  Ideations  in 
Adolescents 

Fadi  T.  Maalouf, 
M.D. 

University  of 
Pittsburgh 

2008 

85,000 

Childhood  Trauma  and 
Suicidal  Behaviors:  The 

Role  of  the  Hypothalamic- 
Pituitary-Adrenal  Axis  in 
Mediating  Risk 

Holly  C.  Wilcox, 
Ph.D. 

Johns  Hopkins 
University 

2008 

85,000 

Family  Therapy  for 

Hospital  Care 

Guy  Diamond, 
Ph.D. 

Children's 
Hospital  of 
Philadelphia 

2008 

85,000 

Suicide  among  Military 
Veterans:  Analysis  of  the 
National  Violent  Death 
Reporting  System 

Mark 

Kaplan, M.P.H., 
Dr.P.H. 

Portland  State 
University 

2008 

85,000 

Glutamatergic  Signaling 
in  the  Locus  Coeruleus  in 
Depression  and  Suicide 

Gregory 

Ordway,  Ph.D. 

East  Tennessee 
State  University 

2008 

85,000 

Explaining  the  Sexual 
Orientation  Disparity  in 
Adolescent  Suicide  Risk 

Stephen  Russell, 
Ph.D. 

University  of 
Arizona 

2008 

74,700 

Suicide  Rates  and  Liang  Zhou, 

Misclassification  of  Suicides  M.D.,  Ph.D. 
in  Hui,  Tibetan  and  Han  in 

Qinghai  Province  of  China 

Xiangya  Medical 
School 

2008 

72,250 

Genetic  Variation  in 
Glutamate  Receptor 
Subunits  May  Contribute 
to  Suicidal  Ideation 
Susceptibility  in  Pregnancy 

Alicia  Smith, 
Ph.D. 

Emory  University 

Participants 


NAASP  Category 


MOMRP  Category 


Civilians 


Civilians 


Civilians 


Civilians 


Civilians 


Civilians 


Civilians 


Civilians 


Prediction  of  imminent  risk 


Improved  biological 
interventions 


Enhanced  continuity  of  care 

Psychosocial  interventions  for 
those  at  risk 

Psychosocial  interventions  for 
those  at  risk 

Risk  and  protective  factor 
interactions 

Improved  biological 
interventions 


Prediction  of  imminent  risk 


Psychosocial  interventions  for 
those  at  risk 

Risk  and  protective  factor 
interactions 

Prediction  of  imminent  risk 
Improved  biological 
interventions 


Early  screening/intervention 
Assessment 


Epidemiology  and/or  basic 
science/neurological  mechanisms 


Treatment 


Prevention,  education,  and 
training 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

Epidemiology  and/or  basic 
science/neurological  mechanisms 


Prevention,  education,  and 
training 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

Prevention,  education,  and 
training 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

Assessment 
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Table  A. 5 — Continued 


Grant  Principal  Researching 

Year  Funding  (S)  Title  Investigator(s)  Organization 


2008 

72,250 

The  Impact  of  RNA  Editing 
on  Suicide  Risk 

Monsheel  Sodhi, 
Ph.D. 

University  of 
Alabama 

2008 

72,250 

Suicide  Risk  and  the 

Danish  Criminal  Justice 
System:  A  Nested  Case- 
Control  Study 

Roger  Webb, 
Ph.D.,  M.Sc., 

M.A. 

University  of 
Manchester 

2008 

72,250 

An  Adjunctive 

Intervention  to  Reduce 
Suicide  Risk  in  Patients 
with  Bipolar  1  Disorder 

Lauren 

Weinstock,  Ph.D. 

Brown  University 

2008 

72,250 

Genome-Wide  Association  Virginia  Willour, 
Study  of  Attempted  Suicide  Ph.D. 

Johns  Hopkins 
University 

2008 

63,750 

Identifying  Early 
Intervention  Targets  for 
College  Students  at  Risk 
for  Suicide 

Amelia  Arria, 
Ph.D. 

University  of 
Maryland 

2008 

63,750 

Expression  of  Truncated 
Isoform  of  Neuronal 
Tryptophan  Hydroxylase  in 
the  Brainstem  of  Depressea 
Suicides 

Helene  Bach, 
Ph.D. 

f 

Columbia 

University 

2008 

63,750 

Reduced  Plasticity  in  the 
Hippocampus  of  Suicide 

Maura  Boldrini, 
M.D.,  Ph.D. 

Columbia 

University 

2008 

63,750 

Does  the  Nature  of 
Treatment  Provided 

Cindy  Claassen, 
Ph.D. 

University  of 
Texas 

After  a  Suicide  Attempt 
Impact  the  Risk  of 
Repetition 


Participants  NAASP  Category  MOMRP  Category 


Civilians 


Civilians 


Civilians 


Civilians 


Civilians 


Improved  biological 
interventions 

Psychosocial  interventions  for 
those  at  risk 

Risk  and  protective  factor 
interactions 

Prediction  of  imminent  risk 

Enhanced  continuity  of  care 
Provider  and  gatekeeper 
training 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions  for 
those  at  risk 

Psychosocial  interventions  for 
those  at  risk 

Prediction  of  imminent  risk 


Epidemiology  and/or  basic 
science/neurological  mechanisms 

Epidemiology  and/or  basic 
science/neurological  mechanisms 


Prevention,  education,  and 
training 

Early  screening/intervention 


Epidemiology  and/or  basic 
science/neurological  mechanisms 

Early  screening/intervention 
Epidemiology  and/or  basic 
science/neurological  mechanisms 


Civilians  Improved  biological 

interventions 


Epidemiology  and/or  basic 
science/neurological  mechanisms 


Civilians 


Civilians 


Psychosocial  interventions  for 
those  at  risk 

Prevention  of  reattempts 


Epidemiology  and/or  basic 
science/neurological  mechanisms 

Postvention 

Epidemiology  and/or  basic 
science/neurological  mechanisms 
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Table  A. 5 — Continued 


Grant  Principal  Researching 

Year  Funding  (S)  Title  Investigator(s)  Organization  Participants  NAASP  Category 


2008 

63,750 

Ionotropic  Glutamate 
Receptor  mRNA  Editing  In 
Suicide 

Stella  Dracheva, 
Ph.D. 

Mount  Sinai 

School  of  Medicine 

Civilians 

Improved  biological 
interventions 

2008 

63,750 

Developing  Precllnical 
Models  of  Suicide 
Endophenotypes  to 
Delineate  Lithium's 
Antisuicidai  Target 

Todd  Gould, 

M.D. 

University  of 
Maryland 

Civilians 

Improved  biological 
interventions 

2008 

63,750 

Transmission  of  Suicidal 
Behavior  in  Three 
Generations  of  the 

Same  Families:  A  30  Year 
Community  Study 

Stephanie 

Kasen,  Ph.D. 

New  York  State 
Psychiatric 

Institute, 

Columbia 

University 

Civilians 

Risk  and  protective  factor 
interactions 

Population-based  risk  reduction/ 
resilience-building 

2008 

63,750 

Suicidality  Among 
Childbearing  Women 

Richard 
Neugebauer, 
M.D.,  Ph.D. 

Columbia 

University 

Civilians 

Risk  and  protective  factor 
interactions 

2008 

63,750 

fMRI  Study  of  Brain 
Circuitry  Underlying 
Impulsivity  and  Affect 
Regulation  in  Suicidal 
Patients  with  Adolescent 
Bipolar  Disorder 

Mani  Pavuluri, 
M.D.,  Ph.D. 

University  of 

Illinois  at  Chicago 

Civilians 

Improved  biological 
interventions 

2008 

63,750 

Using  a  Laboratory 
Measure  of  Emotion 
Regulation  to  Characterize 
Suicide  Attempters 

Zachary 

Rosenthal,  Ph.D. 

Duke  University 

Civilians 

Improved  biological 
interventions 

2008 

63,750 

Determining  the  Effects 
of  Early  Alcohol  Use  on 

Monica  Swahn, 
Ph.D.,  M.P.H. 

Georgia  State 
University 

Civilians 

Risk  and  protective  factor 
interactions 

Suicide  Attempts  in  Early 
Adolescence  through 
Young  Adolescence 


MOMRP  Category 


Epidemiology  and/or  basic 
science/neurological  mechanisms 


Epidemiology  and/or  basic 
science/neurological  mechanisms 


Prevention,  education,  and 
training 

Assessment 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

Epidemiology  and/or  basic 
science/neurological  mechanisms 


Epidemiology  and/or  basic 
science/neurological  mechanisms 


Epidemiology  and/or  basic 
science/neurological  mechanisms 


Prevention,  education,  and 
training 

Epidemiology  and/or  basic 
science/neurological  mechanisms 
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Table  A. 5 — Continued 


Grant  Principal  Researching 


Year 

Funding  (S) 

Title 

Investigator(s) 

Organization 

Participants 

NAASP  Category 

MOMRP  Category 

2008 

59,790 

A  Prospective  Study  of 
the  Relationship  Between 
Non-Suicidal  Self-Injury 
and  Suicidal  Behavior  In  a 
College  Population 

Janis  Whitlock, 
Ph.D.,  M.P.H. 

Cornell  University 

Civilians 

Risk  and  protective  factor 
interactions 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

2008 

20,000 

Investigating  Glial  Cells  Naguib 

Line-Derived  Mechawar,  Ph.D. 

Neurotrophic  Factor  (GDNF) 
in  the  Amygdala  of  Suicide 

Victims 

McGill  University 

Civilians 

Improved  biological 
interventions 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

2008 

20,000 

Factors  Promoting  Good 
Adult  Functioning  in 
Adolescents  with  Suicidal 
Ideation:  A  Pilot  Study 

Helen  Reinherz, 
Sc.D. 

Simmons  College 

Civilians 

Provider  and  gatekeeper 
training 

Psychosocial  interventions  for 
those  at  risk 

Prevention,  education,  and 
training 

Early  screening/intervention 

Prediction  of  imminent  risk 

2008 

20,000 

Assessing  Implicit  Self- 
Injurious  Behavior  in 

Police  Officers 

John  Violanti, 
M.D.,  Ph.D. 

State  University 
of  New  York  at 
Buffalo 

Civilians 

Prediction  of  imminent  risk 

Assessment 

2007 

99,321 

Culturally  Competent 
Family-Based  Intervention 
Versus  Enhanced  Usual 

Care  In  the  Community  for 

Nadine  Kaslow, 
Ph.D. 

Emory  University 

Civilians 

Affordable,  accessible,  and 
effective  care 

Psychosocial  interventions  for 
those  at  risk 

Treatment 

Epidemiology  and/or  basic 
science/neurological  mechanisms 

Reducing  Psychological 

Distress  and  Enhancing  R'sk  and  protective  factor 

Functioning  in  Suicidally  interactions 

Bereaved  African 
Americans 
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APPENDIX  B 


Alternative  Allocation  Analysis 


In  Chapter  Two,  we  allocated  funds  for  interventions  across  multiple  categories,  resulting  in 
“double-counting.”  An  alternative  strategy  would  be  to  allocate  funds  equally  across  categories. 
We  conducted  a  secondary  analysis  using  this  method.  The  results  are  presented  in  Tables  B.l 
and  B.2. 

Table  B.1 

Funding  Totals  for  Each  NAASP  Category,  Assuming  Dispersed  Equal  Allocations 


NAASP  Category 

Funding  ($) 

Overall 

DoD 

VA 

DHHS 

AFSP 

Psychosocial  interventions 

36,106,203 

27,150,147 

1,399,937 

5,891,984 

1,664,135 

Prediction  of  imminent  risk 

15,319,954 

4,732,877 

1,860,590 

7,684,702 

1,041,785 

Risk  and  protective  factors 

32,259,925 

18,461,386 

2,292,637 

10,125,437 

1,380,466 

Biological  interventions 

11,454,261 

5,826,035 

627,747 

3,312,063 

1,688,417 

Accessible  and  effective  care 

21,907,716 

20,536,585 

0 

1,136,205 

234,925 

Enhanced  continuity  of  care 

13,146,012 

6,118,409 

991,661 

5,461,954 

573,989 

Population-based  risk-reduction 

12,515,246 

9,735,432 

387,919 

2,233,937 

157,958 

Provider/gatekeeper  training 

7,988,608 

5,973,327 

0 

1,783,743 

231,538 

Population-based  screening 

2,944,440 

989,941 

435,894 

1,518,605 

0 

Prevention  of  reattempts 

5,834,140 

2,803,619 

0 

2,856,453 

174,068 

Stigma  reduction 

6,003,066 

4,946,749 

0 

988,818 

67,500 

Access  to  lethal  means 

351,186 

0 

129,237 

221,949 

0 
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Table  B.2 

Funding  Totals  for  Each  MOMRP  Category,  Assuming  Dispersed  Equal  Allocations 


Funding  ($) 

MOMRP  Category 

Overall 

DoD 

VA 

DHHS 

AFSP 

Prevention,  education,  and  training 

22,102,217 

17,455,846 

2,011,117 

2,635,255 

1,015,804 

Early  screening  and  intervention 

17,052,808 

3,740,201 

0 

13,312,607 

456,508 

Assessment 

14,632,933 

6,239,801 

2,375,835 

6,017,297 

722,547 

Treatment 

53,910,793 

43,160,528 

777,863 

9,972,402 

1,135,418 

Recovery  and  return  to  duty 

42,500 

0 

0 

0 

42,500 

Postvention 

4,857,287 

4,857,287 

0 

0 

164,267 

Epidemiology  and/or  basic  science/ 
neurological  mechanisms 

46,059,939 

31,820,842 

2,960,808 

11,278,289 

3,677,738 

APPENDIX  C 


Statistical  Procedure  for  Extracting  Rankings  from  the  RAND 
ExpertLens  Panel 


Bayesian  Modeling 


We  employed  a  Bayesian  framework  to  analyze  the  data  derived  from  the  ExpertLens  elicita¬ 
tion  process.  The  framework  treats  model  parameters  as  unknown,  random  variables  on  which 
we  imposed  prior  distributions  that  were  updated  or  “learned”  by  the  data.  For  example,  if  our 
model  holds  one  parameter,  Q,  let  us  assign  a  prior  distribution,  to  model  the  uncer¬ 

tainty  for  0-  The  data  will  “learn”  the  value  for  by  updating  our  prior  distribution  to  a  poste¬ 
rior  distribution,  n[6  \  y),  using  Bayes’  theorem: 


n{e\y) 


where  y  denotes  the  sample  data.  We  choose  a  Bayesian  approach  for  our  work  because  it 
allowed  us  to  introduce  a  latent  continuous  response  in  generating  our  observed  ordered  scores, 
an  intuitive  formulation  that  provides  for  useful  inference. 


Model  to  Extract  Intrinsic  Goal  Score 

We  introduced  a  latent  response,  t-j,  where  z  =  1, .  .  .  ,  G  =  60  indexes  a  goal-criterion  combina¬ 
tion,  such  as  the  importance  of  population-based  risk  reduction/resilience-building  in  reducing 
the  military  suicide  rate.  We  evaluated  each  of  the  goals  according  to  hve  criteria  to  produce 
the  G  =  60  combination.  Here,  we  refer  to  the  goal-criterion  combination  with  the  shorthand 
label  “goals  when  discussing  the  model  and  the  results  from  the  expert  elicitation.”  The  index, 
j  =  I,  J  =  28,  denotes  participating  raters,  where  rater  provides  an  ordered  score, 
for  goal  z.  Each  t-j  takes  values  on  a  continuous  scale  and  is  associated  with  our  observed 
ordered  category  scores,  y.j .  We  can  interpret  t-j  as  the  latent  propensity  for  rater  j,  scoring 
goal  z,  to  be  in  the  category  associated  with  the  observed  (Hausman  and  Wise,  1978).  We 
formally  make  the  “link”  between  and  y^j  by  introducing  a  set  of  random  cut  points,  y, 
that  divide  the  support  (or  values  of)  t.j  into  categories  associated  with  More  specifically, 
we  restrict  yf  —  1  <  <  ji  when  the  associated  y^  =  L  Figure  C.l  provides  a  conceptual  illus¬ 

tration  of  how  portions  of  the  probability  mass  for  the  latent  response,  t,  are  allocated  to  each 
observed  score  category,  y  =  f  for  f  =  (1, . .  .  ,  Q  ordered  categories,  based  on  the  cut  points,  y . 
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Figure  C.1 

7  "Chops"  Up  Probability  Mass  of  f  to  Compute 


t 


RAND  RR559-C 1 


Prior  research  (Johnson,  1996;  Johnson  and  Albert,  1999;  Savitsky  and  Dalai,  2014)  spec¬ 
ifies  a  Bayesian  parametric  approach  for  the  analysis  of  multirater  ordered  data  that  defines  the 
regression  construction 


where  the  likelihood  for  t.j  is  induced  from  the  observed  ordered  Jij  through  y.  This  formula¬ 
tion  possesses  the  feature  that  precision  parameters,  v ^  are  specified  by  rater  to  differentially 
denoise  or  weight  the  multiple  ratings  to  reveal  an  intrinsic  latent  “true”  score,  for  goal 
i  e(l,  .  .  .  ,  G).  Rater with  estimated  precision,  v  ■  >  v p  will  be  weighted  more  highly  than 
rater  j'  in  determining  g.  Raters  who  express  higher-precision  values  are  viewed  as 

possessing  more  expertise  or  being  more  engaged  in  the  elicitation  process.  This  construction 
achieves  both  denoising  and  differential  weighting  of  raters  based  on  the  precisions  that  encode 
their  expertise. 

Each  panel  in  Figure  C.2  presents  the  distribution  of  observed  scores,  y^.  =  ya)’ 

for  randomly  selected  raters,  z  e  (1,  .  .  .  ,  /  =  28),  to  provide  intuition  on  the  precision  of  raters’ 
scores.  We  observe  that  raters  Expert  14  and  Expert  86  express  relatively  diffuse  distribu¬ 
tions  for  their  raw  scores  over  the  goals,  while  Expert  16  and  Expert  89  are  relatively  more 
concentrated. 

The  precision  parameters  receive  the  prior  formulation. 
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Figure  C.2 

Distributions  of  Raw  Scores  for  Selected  Raters 
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where  shape  hyperparameter,  a^,  and  rate  hyperparameter,  are  each  drawn  from  a  further 
U (1,25)  prior.  The  cut  points,  are  random  and  specihed  with  priors: 


We  employed  the  construction  from  Ishwaran  (2000)  to  improve  mixing  in  posterior 
samplings  for  y j  that  specifies 
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for  = 

The  constant,  C  =  10,  denotes  the  number  of  ordered  categories  for  each  observed  y^. 
This  construction  allows  the  (pj  to  be  sampled  without  order  restrictions,  and  the  exponentia¬ 
tion  inflates  the  prior  scale  for  Yj,  which  could  improve  mixing. 

Finally,  we  set  the  prior  for  the  intrinsic  goal  scores: 

iid 

z.  ~A^(0,1), 

where  we  fix  the  scale  to  1  to  identify  the  scale  for  the  model  because  ordered  data  do  not 
convey  scale  information. 


Results 

We  extracted  posterior  distributions  for  the  intrinsic  goal  scores,  z-,  after  collapsing  overraters 
for  each  criterion.  The  relative  differences  between  the  z-,i  =  ,  G  =  60  may  be  used  to 

rank  their  performance  for  each  criterion.  In  particular,  each  panel  in  Figure  C.3  presents  the 
posterior  distributions  of  the  intrinsic  scores  for  each  of  the  goals.  While  the  relative  rankings 
differ  somewhat  across  criteria,  we  note  that  four  of  the  goals  that  consistently  rank  highly  are 
(1)  gatekeeper  training,  (2)  continuity  of  care,  (3)  high-quality  care,  and  (4)  provider  training. 
These  top-ranked  goals  focus  on  aspects  of  treatment  delivery  rather  than  the  root  causes  of 
suicidal  ideation  or  differential  approaches  to  screening.  The  gatekeeper  training  goal  includes 
identifying  at-risk  service  members  such  that  there  may  be  an  implicit  preference  for  human- 
based  screening  performed  by  those  closest  to  service  members,  rather  than  more  mechanical 
or  technological  alternatives.  It  also  bears  mentioning  that  25  of^'  =  28  participating  raters  were 
not  primarily  researchers,  so  there  may  be  more  a  preference  for  exploring  treatment  modalities 
rather  than  root  causes. 

Figure  C.4  presents  the  posterior  distributions  for  the  rater  precisions,  Vj,  for  each  of  the 
raters  included  in  our  expert  elicitation,  ordered  from  highest  to  lowest  posterior  mean  preci¬ 
sion.  We  recall  that  higher  precisions  connote  more  expertise  and  more  engagement  with  the 
process,  while  low  values  connote  the  opposite.  The  precision  distributions  express  a  high  gra¬ 
dient  or  variation  among  the  raters,  with  Expert  16  producing  among  the  highest  values  and 
Expert  14  the  lowest.  So,  we  are  wise  to  differentially  weight  rater  contributions  in  determining 
the  intrinsic  goal  scores,  z^. 


Figure  C.3 

Posterior  Distributions  of  the  Intrinsic  Goal  Scores,  z. 
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Figure  C.4 
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APPENDIX  D 


Feedback  from  DoD  Stakeholders  About  the  RAND 
ExpertLens  Process 


At  the  end  of  round  3  of  the  RAND  ExpertLens  elicitation,  participants  were  asked  a  series 
of  questions  about  their  perceptions  of  the  online  process.  For  the  most  part,  participants  had 
positive  feedback,  as  shown  in  Table  D.l. 

Table  D.1 

DoD  Stakeholder  Feedback  on  the  RAND  ExpertLens  Process 


%  Agreeing  or 

Feedback  Prompt  Strongly  Agreeing 

A  small  number  of  people  dominated  the  discussions.  20 

The  discussions  gave  me  a  better  understanding  of  the  issues.  60 

This  study  was  too  long.  10 

I  had  trouble  following  the  discussions.  0 

I  was  reluctant  to  share  some  of  my  views  during  the  discussion.  5 

The  ExpertLens  was  easy  to  use.  60 

Participants  debated  each  other's  viewpoints  during  the  discussion.  5 

Participation  in  this  study  was  frustrating.  0 

The  discussions  brought  out  views  I  hadn't  considered.  15 

Participation  in  this  study  took  a  lot  of  effort.  10 

The  discussions  brought  out  divergent  views.  25 

Participants  sometimes  misinterpreted  each  other's  comments  5 

during  the  discussion. 

Participation  in  this  study  was  interesting  35 

The  discussion  round  caused  me  to  revise  my  original  answers.  20 

I  was  comfortable  expressing  my  views  in  the  discussion  round.  40 

The  right  set  of  questions  was  asked  in  this  study.  30 

My  expertise/experience  is  relevant  to  the  topic  of  study.  75 

I  would  like  to  use  ExpertLens  in  the  future.  45 
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Sensitivity  Analysis  of  the  Benefit-Cost  Index  Rankings 


This  appendix  explores  the  robustness  of  the  benefit-cost  index  when  presented  in  terms  of 
ordinal  rankings.  We  first  discuss  how  an  error  in  any  parameter  value  would  affect  the  overall 
index  value.  We  then  quantify  the  extent  to  which  index  values  may  change  without  affecting 
ordinal  rankings. 


Index  Sensitivity  to  Errors  in  Parameters 

suicides  prevented  nY-  pYsYi 

index  = - = - . 

cost  c 

Based  on  the  formula  for  the  index,  it  is  straightforward  to  infer  how  a  percentage  error 
or  percentage  change  in  the  parameters  will  affect  the  index.  Any  change  in  the  numerator  will 
induce  an  equal  change  in  the  index.  For  example,  a  5 -percent  individual  error  in  any  of  the 
four  variables  or  a  5 -percent  combined  error  in  the  variables  in  the  numerator  will  translate  to 
a  5-percent  error  in  the  index  (see  Table  E.l).  Because  cost  is  in  the  denominator,  a  percentage 
error  in  the  denominator  requires  taking  the  inverse.  Table  E.l  shows  how  errors  in  the  param¬ 
eters  translate  to  errors  in  the  index. 


Robustness  of  Ordinal  Rankings 

While  the  index  values  are  directly  sensitive  to  errors  in  the  parameters  that  compose  the 
index,  we  are  less  concerned  with  the  cardinal  value  of  the  index  than  with  the  ordinal  value 

Table  E.l 

Percentage  Change  in  the  Benefit-Cost  Index,  by  Variable 


Variable  Percentage  Change 


n  X  p  X  s  X  /■ 

-50 

-20 

-10 

-5  -1 

0 

1 

5 

10 

20 

50 

Index 

-50 

-20 

-10 

-5  -1 

0 

1 

5 

10 

20 

50 

c 

-50 

-20 

-10 

-5  -1 

0 

1 

5 

10 

20 

50 

Index 

100 

25 

11 

5  1 

0 

-1 

-5 

-9 

-17 

-33 
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(i.e.,  the  rankings  of  the  indexes).  Table  E.2  shows  the  indexes  we  calculated  for  each  aspira- 
tional  research  goal.  The  percentage  “lead”  over  the  next-lower  goal  is  the  absolute  value  of  the 
percentage  change  from  that  goal  to  the  next  goal  below.  Conceptually,  the  percentage  column 
shows  the  relative  margin  of  error.  For  example,  the  difference  between  provider  training  and 
help-seeking  is  47  percent.  Thus,  our  estimates  for  provider  training  and  help-seeking  could  be 
inaccurate,  but  the  provider  training  index  would  have  to  be  47-percent  too  high,  the  help¬ 
seeking  index  would  have  to  be  47  percent  too  low,  or  both  would  have  to  vary  to  a  propor¬ 
tional  degree  for  the  ordinal  rankings  to  change.  For  certain  goals  {reduced  access,  psychosocial 
interventions,  risk  reduction,  and  quality  care),  the  relative  margins  are  close,  so  a  small  change 
(<  15  percent)  would  affect  the  rankings.  For  example,  if  the  risk  reduction  index  decreased  by 
more  than  1  percent,  it  would  fall  behind  quality  care  in  the  rankings. 

Another  way  to  see  the  margin  for  the  estimation  error  is  to  graph  the  log(index)  for  each 
goal.  In  Figure  E.l,  differences  in  the  heights  of  each  bar  are  expressed  in  terms  of  percentage 
changes.  We  can  see  that  only  a  large  error  in  estimating  the  parameters  would  cause  the  order 
of  the  blue  bars  to  change,  but  among  the  red  bars,  all  of  which  have  roughly  the  same  heights, 
a  small  change  would  result  in  slightly  reordered  rankings. 

To  illustrate,  help-seeking  has  a  significant  relative  margin  from  both  the  next-higher  and 
next-lower  goals.  We  originally  assumed  that  the  per-person  cost  of  implementing  help-seeking 
was  $1  based  on  the  average  historical  anti-stigma  campaign  expenditure  (Evans-Eacko  et  ah, 
2013).  The  reference  also  includes  a  minimum  expenditure  of  $0.60  per  person  and  a  maxi¬ 
mum  expenditure  of  $1.50  per  person.  If  we  recalculate  the  help-seeking  index  using  either  the 
minimum  or  maximum  expenditure,  we  find  that  the  change  in  the  cost  is  not  enough  to  dis¬ 
place  help-seeking  as  the  second-ranked  goal. 

Therefore,  based  on  Figure  E.l,  we  would  generally  expect  the  highest-  and  lowest-ranked 
research  goals  to  maintain  their  rankings,  given  reasonable  errors  in  the  assumptions  underly- 


Table  E.2 

Percentage  Change  in  Benefit-Cost  Index,  by  Goal 


Aspirational  Goal 

Benefit-Cost 

Index 

%  "Lead"  Over 
Next  Lower 

Provider  training 

93.333 

47 

Help-seeking 

49.558 

96 

Reduced  access 

1.800 

7 

Gatekeeper  training 

1.667 

33 

Prevent  reattempts 

1.125 

35 

Psychosocial  interventions 

0.733 

8 

Risk  reduction 

0.672 

1 

Quality  care 

0.667 

10 

Continuity  of  care 

0.600 

33 

Screening 

0.400 

67 

Biological  interventions 

0.133 

63 

Prediction 

0.050 

— 
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ing  their  indexes.  However,  several  of  the  midranked  goals  have  indexes  that  are  relatively  close 
together,  and  we  could  see  reordered  rankings  in  the  middle  given  a  different  set  of  parameter 
assumptions. 

Figure  E.1 

Logarithmic  Scale  of  Suicides  Prevented  per  Cost 
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APPENDIX  F 


Research  Domains  and  Approaches  for  Assessing 
Research  Quality 


This  appendix  describes  in  detail  the  types  of  research  domains  and  research  approaches  rel¬ 
evant  to  evaluating  quality  research.  This  appendix  should  be  referenced  in  conjunction  with 
the  guidelines  for  identifying  quality  research  as  discussed  in  the  section  “Evidence  of  Research 
Quality”  (the  second  component  for  effectively  diffusing  research  into  practice)  in  Chapter  Six. 


Relevant  Research  Domains 

Suicide  research  can  be  generated  in  many  research  domains  and  is  being  performed  in  several 
different  disciplines  and  areas  of  biomedical  and  social  sciences. 

Biomedical  Research 

Biomedical  research  is  generally  conducted  in  the  areas  of  psychiatry,  epidemiology,  and  public 
health.  Psychiatry  research  attempts  to  understand  human  behavior  through  both  observations 
of  people  and  an  understanding  of  the  biomedical  factors  that  contribute  to  adaptive  and  mal¬ 
adaptive  behavior  and  to  modify  behavior  through  behavioral  or  medical  interventions.  Epi¬ 
demiology  research  attempts  to  identify  the  underlying  patterns  or  causes  of  health  issues  and 
diseases  within  a  broader  population  or  within  a  specific  group.  Public  health  research  seeks  to 
inform  policy  decisions  to  improve  the  health  of  individuals. 

Social  Science  Research 

Social  science  research  is  conducted  in  three  broad  areas: 

1.  Psychological/behavioral  health  (e.g,  clinical  psychology,  counseling  psychology,  clinical  social 
work).  Generally,  this  type  of  research  attempts  to  understand  human  behavior  through 
both  observations  and  experiments  that  typically  have  less  of  a  biomedical  focus  than 
psychiatric  research  and  to  modify  behavior  or  cognitions  through  behavioral  methods. 

2.  Sociocultural  (e.g,  sociology,  social  psychology,  developmental  psychology,  family  systems). 
In  this  domain,  sociological  research  attempts  to  improve  society  by  examining  soci¬ 
etal  processes  and  sociocultural  groups  (e.g.,  race/ethnicity).  Social  psychology  research 
focuses  on  understanding  individual  differences  in  behaviors  within  and  between 
sociocultural  groups.  Developmental  psychology  looks  at  the  behavior  of  individuals 
or  groups  over  a  lifespan  or  targets  one  specific  developmental  period  (e.g.,  childhood). 
Family  systems  research  seeks  to  understand  human  behavior  by  examining  an  indi¬ 
vidual  influenced  by  the  patterns  of  behavior  within  families. 
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3.  Holistic  approaches  attempt  to  reach  across  disciplines  and  settings  to  understand  behav¬ 
ior  on  a  broader  level.  Such  approaches  may  incorporate  community  partners  to  exam¬ 
ine  a  new  idea  and  can  spread  to  areas,  such  as  spirituality  and  religion,  to  understand 
behavior. 


Types  of  Research  Approaches 

Studies  are  typically  cross-sectional  or  longitudinal  in  design.  Cross-sectional  studies  look  at 
groups  or  individuals  at  one  point  in  time.  Because  variables  are  measured  and  compared  at 
the  same  point  in  time,  cross-sectional  studies  cannot  imply  causality.  Longitudinal  studies,  in 
which  one  variable  is  measured  at  time  A  and  another  is  measured  at  time  B,  are  stronger  in 
this  sense  because  they  are  better  able  to  suggest  causation.  That  is,  they  can  reveal  that  the 
variable  measured  at  time  A  affected  the  occurrence  at  time  B. 

Studies  are  designed  with  varying  levels  of  rigor  and  are  classified  as  having  experimental 
or  nonexperimental  designs: 

•  Experimental  designs  manipulate  something  in  the  environment  to  look  at  the  effects  on 
behavior.  Experimental  studies  are  typically  longitudinal  in  design: 

-  RCTs  are  the  standard  for  high-quality  experimental  study.  These  trials  involve  ran¬ 
domly  assigning  individuals  to  a  treatment  condition  (e.g.,  an  intervention  to  reduce 
depression)  or  a  control  condition  (e.g.,  no  treatment  or  treatment  as  usual)  to  deter¬ 
mine  whether  reductions  (or  improvements)  in  the  targeted  outcomes  of  the  treatment 
are  signihcantly  better  for  those  who  received  the  treatment  compared  with  those  who 
did  not. 

-  Group  randomized  studies  randomly  assign  groups  to  conditions  and  evaluate  differ¬ 
ences  between  these  conditions  based  on  group  membership.  These  studies  are  typi¬ 
cally  conducted  when  randomly  assigning  individuals  (as  opposed  to  groups)  may  be 
difficult. 

-  Quasi-experimental  designs  include  no  randomization.  This  process  is  sometimes  used 
when  it  is  difficult  to  randomly  assign  participants  because  they  are  already  receiving 
treatment. 

-  A  mixed-methods  paradigm  involves  the  use  of  case  studies  (i.e.,  qualitative  research) 
in  the  midst  of  an  RCT  (Dattilio,  Edwards,  and  Eishman,  2010).  Case  studies  are 
likely  much  more  accessible  to  providers  and  personnel  working  with  service  members 
because  these  personnel  can  relate  the  experience  of  a  case  target  with  their  own  experi¬ 
ences  with  service  members. 

•  Nonexperimental  designs  include  surveys,  meta- analyses,  observational  studies,  case  stud¬ 
ies,  program  evaluations,  and  studies  using  existing  data  for  the  secondary  analyses  of 
research  ideas.  While  not  all  as  rigorous  in  design  as  experimental  studies,  all  of  these 
studies  have  implications  for  informing  practice  in  biomedical  and  social  science  settings. 
Nonexperimental  studies  can  either  be  longitudinal  or  cross-sectional: 

-  Survey  research  involves  distributing  questionnaires  in  person,  by  mail,  by  phone,  by 
computer,  or  over  the  Internet  to  solicit  responses  from  a  population  of  interest. 
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-  Meta-analyses  look  at  the  results  reported  in  prior  studies  to  make  conclusions  about  the 
potential  effectiveness  of  that  body  of  research  (e.g.,  analyses  of  all  published  research 
on  RCTs  in  a  specific  area). 

-  Observational  studies  use  methods  to  document  overt  observations  of  human  behavior 
in  natural  settings  or  in  contrived  research  or  laboratory  settings. 

-  Comparison  studies  (pre-post  designs)  use  nonrandomized  group  data  to  compare  dif¬ 
ferences  in  outcomes  between  individuals  and  between  groups  or  changes  over  time 
within  individuals  and  within  groups. 

-  Case  studies  provide  important  information  about  behavior.  In  clinical  settings,  they 
are  typically  a  description  of  one  individual’s  unique  behavior  or  describe  an  individu¬ 
al’s  reaction  to  a  treatment  or  program. 

-  Program  evaluations  are  studies  that  look  at  the  effects  of  an  established  program  on 
behavior  or  public  health  policy. 

-  Studies  examining  existing  data  for  secondary  analysis  purposes  can  answer  other  research 
questions  that  may  come  about  after  a  study  is  completed. 


Types  of  Research  Findings 

Two  categories  of  relevant  research  results  are  applicable  to  military  suicide.  Both  qualitative 
and  quantitative  results  yield  important  research  data: 

•  Qualitative  results  from  research  studies  typically  involve  exploratory  methods  with  gen¬ 
eral  flexibility  in  the  research  questions.  It  describes  variation,  relationships,  and  indi¬ 
vidual/group  experiences,  typically  through  open-ended  responses  from  interviewees  or 
focus-group  attendees.  Researchers  often  analyze  qualitative  results  by  categorizing  the 
data  into  themes.  The  data  are  typically  reviewed  by  several  members  of  a  research  team, 
who  agree  to  the  categorizations. 

•  Quantitative  results  from  research  studies  typically  test  specific  hypotheses  that  have  been 
established  since  the  inception  of  the  study  (a  priori).  Quantitative  results  are  typically 
analyzed  using  rigorous  statistical  procedures. 
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